Storage medium having stored thereon game program, game apparatus, game system, and game processing method

ABSTRACT

On the basis of data based on an attitude and/or a motion of a portable display apparatus body, an action of a player object placed in a virtual world is controlled, and on the basis of a position and/or an attitude of the player object, it is determined whether or not the player object has received an attack. Further, when the player object has received a predetermined attack, an attack effect image representing an effect of the predetermined attack is generated, and a first image is generated by superimposing the attack effect image on an image of the virtual world and displayed on the portable display apparatus. Then, the effect of the predetermined attack in an area in the attack effect image is repaired, the area overlapping a predetermined range whose center is a touch position on a touch panel.

CROSS REFERENCE TO RELATED APPLICATION

The disclosures of Japanese Patent Application No. 2011-050039, filed onMar. 8, 2011, Japanese Patent Application No. 2011-083453, JapanesePatent Application No. 2011-083454, Japanese Patent Application No.2011-083455, and Japanese Patent Application No. 2011-083456, filed onApr. 5, 2011, Japanese Patent Application No. 2011-115402, JapanesePatent Application No. 2011-115403, and Japanese Patent Application No.2011-115404, filed on May 24, 2011, Japanese Patent Application No.2011-118901 and Japanese Patent Application No. 2011-118902, filed onMay 27, 2011, Japanese Patent Application No. 2011-123644, JapanesePatent Application No. 2011-123645, and Japanese Patent Application No.2011-123646, filed on Jun. 1, 2011, and Japanese Patent Application No.2011-225538, filed on Oct. 13, 2011, are incorporated herein byreference.

FIELD

The technique shown here relates to a storage medium having storedthereon a game program, a game apparatus, a game system, and a gameprocessing method, and in particular, relates to a storage medium havingstored thereon a game program that, for example, performs processingbased on the attitude and/or the motion of a display apparatus, and agame apparatus, a game system, and a game processing method that, forexample, perform processing based on the attitude and/or the motion of adisplay apparatus and the action of a user.

BACKGROUND AND SUMMARY

Conventionally, there is a game where shooting is performed at a target,displayed on a display apparatus, using a shooting device. A gameapparatus for performing the shooting game includes a shooting devicefor performing shooting, and attack/defense input means for selectingand inputting an attack state or a defense state. In the shooting gameapparatus, when a player has operated the attack/defense input means toselect the attack state, shooting is performed, on the basis of an inputfrom the shooting device, at an enemy that appears in a game space. Whenthe player has operated the attack/defense input means to select thedefense state, a virtual player in the game space is caused to take adanger avoidance action.

In the attack/defense input means of the shooting game apparatus,however, the defense state is selected as a result of the playerstepping on a pedal. Accordingly, such an operation is totally differentfrom and far removed from the danger avoidance action to be taken by thevirtual player. This may cause the player performing the operation tolose the impression of being in a virtual world.

Therefore, it is an object of an exemplary embodiment to provide astorage medium having stored thereon a game program capable of improvingvirtual reality by causing a player object in a virtual world to takeaction in accordance with an operation of a user, and a game apparatus,a game system, and a game processing method that are capable ofimproving virtual reality by causing a player object in a virtual worldto take action in accordance with an operation of a user.

To achieve the above object, the exemplary embodiment may employ, forexample, the following configurations. It is understood that when thedescription of the scope of the appended claims is interpreted, thescope should be interpreted only by the description of the scope of theappended claims. If the description of the scope of the appended claimscontradicts the description of these columns, the description of thescope of the appended claims has priority.

In an exemplary configuration of a computer-readable storage mediumhaving stored thereon a game program according to the exemplaryembodiment, a game program is executed by a computer of a game apparatuscapable of displaying an image on a portable display apparatus thatoutputs at least body state data based on an attitude and/or a motion ofthe portable display apparatus body and touch position data based on atouch position on a touch panel provided on a surface of a displayscreen of the portable display apparatus. The game program causing thecomputer to execute: controlling, on the basis of the body state data,an action of a player object placed in a virtual world; determining, onthe basis of a position and/or an attitude of the player object, whetheror not the player object has received an attack from another object;generating, when the player object has received a predetermined attackfrom another object, an attack effect image representing an effect ofthe predetermined attack; generating a first image by superimposing theattack effect image on an image of the virtual world viewed from theplayer object, or on an image of the virtual world including at leastthe player object; displaying the first image on the portable displayapparatus; and repairing, when the attack effect image is displayed onthe display screen, the effect of the predetermined attack in an area inthe attack effect image in accordance with a touch operation performedon the touch panel, the area overlapping a predetermined range whosecenter is the touch position indicated by the touch position data.

It should be noted that the game apparatus may be an apparatus thatperforms game processing and generates an image based on the gameprocessing, or may be a versatile apparatus such as a general personalcomputer. The portable display apparatus may have a size small enough tobe carried by a user. Typically, the portable display apparatus may be adisplay apparatus that allows the user to view an image displayedthereon by holding it with both hands. Further, as in a terminalapparatus according to the embodiment described later, the portabledisplay apparatus may or may not include components other than: meansfor outputting at least data based on the attitude and/or the motion ofthe portable display apparatus body; means for outputting touch positiondata based on a touch position on a touch panel provided on a surface ofa display screen of the portable display apparatus; and a display screenthat displays the first image.

Based on the above, it is possible to control the action of a playerobject on the basis of the attitude and the motion of a portable displayapparatus, and therefore possible to cause the player object to take theaction of avoiding an attack from another object on the basis of theattitude and the motion of the portable display apparatus. This makes itpossible to provide a user, performing the above operation, with afeeling as if being in a virtual world. Further, an attack effect imageto be displayed on a display apparatus of the portable display apparatuswhen a predetermined attack has been received makes it possible torepair the effect of the predetermined attack by performing a touchoperation. Accordingly, when an attack has been received, the effect ofthe attack is displayed on the display apparatus of the portable displayapparatus that is used to avoid attacks, and it is also possible torepair the effect by touching an image representing the effect of thereceived attack. This makes it possible to provide the user, operatingthe portable display apparatus, with a feeling as if having received anattack in the virtual world, and also provide the user with a feeling asif actually repairing the effect of the attack received in the virtualworld. The attack avoidance operation, the display when an attack hasbeen received, and the operation of repairing the effect of the receivedattack can be thus performed successively, whereby it is possible toimprove virtual reality dramatically.

In addition, the game program may further cause the computer to executecalculating an attitude and/or a motion of the portable displayapparatus on the basis of the body state data. In this case, at leastone of a direction, the attitude, and the position of the player objectin the virtual world may be controlled on the basis of the calculatedattitude and/or motion of the portable display apparatus.

Based on the above, it is possible to calculate the attitude and/or themotion of the portable display apparatus using body state data outputfrom the portable display apparatus, and control the action of theplayer object (at least one of the direction, the attitude, and theposition of the player object) on the basis of the attitude and/or themotion of the portable display apparatus.

In addition, when the player object and another object have collidedwith each other in the virtual world, it may be determined, on the basisof the direction, the attitude, and the position of the player object inthe virtual world, that the player object has received an attack.

Based on the above, the player object can avoid an attack on the basisof the direction, the attitude, and the position of the player object.This enables the operation of avoiding an attack on the basis of theattitude and the motion of the portable display apparatus capable ofchanging at least one of the direction, the attitude, and the positionof the player object.

In addition, when a specific portion of the player object has collidedwith another object in the virtual world, it may be determined, on thebasis of the direction, the attitude, and the position of the playerobject in the virtual world, that the player object has received anattack. When the player object has received the attack at the specificportion, the attack effect image representing an effect of the attackmay be generated.

Based on the above, for example, a specific portion may be set withrespect to the direction of the player object such as the facial surfaceor the front surface of the player object, whereby it is possible toavoid an attack on the basis of the facing direction of the playerobject even when another object has collided with the player object.This enables the attack avoidance operation even when the player objectis caused to rotate on the basis of the attitude and the motion of theportable display apparatus.

In addition, the action of the player object may be controlled such thatthe player object rotates or moves in accordance with an angle by whicha direction of the portable display apparatus body is changed.

Based on the above, the user can control the action of the player objectby changing the direction of the portable display apparatus.

In addition, the action of the player object may be controlled such thatthe greater a change in the angle by which the direction of the portabledisplay apparatus body is changed, the greater the player object rotatesor moves.

Based on the above, the user can control the amount of movement(including the amount of rotational movement) of the player object onthe basis of the amount of change in the attitude and/or the position ofthe portable display apparatus.

In addition, the attitude and/or the motion of the portable displayapparatus may be calculated with respect to a predetermined direction inreal space. On the basis of the attitude and/or the motion of theportable display apparatus with respect to the predetermined directionin real space, at least one of the direction, the attitude, and theposition of the player object may be controlled with respect to adirection that corresponds to the predetermined direction and is set inthe virtual world.

Based on the above, on the basis of the attitude and/or the motion ofthe portable display apparatus with respect to a predetermined directionin real space, it is possible to control at least one of the direction,the attitude, and the position of the player object with respect to apredetermined direction that corresponds to the predetermined directionin real space and is included in the virtual world.

In addition, an attitude and/or a motion of the portable displayapparatus may be calculated with respect to a direction of gravity inreal space, using the direction of gravity as the predetermineddirection. On the basis of the attitude and/or the motion of theportable display apparatus with respect to the direction of gravity inreal space, at least one of the direction, the attitude, and theposition of the player object may be controlled with respect to adirection corresponding to a direction of gravity set in the virtualworld.

Based on the above, on the basis of the attitude and/or the motion ofthe portable display apparatus with respect to the direction of gravity,it is possible to control at least one of the direction, the attitude,and the position of the player object with respect to the same directionof gravity.

In addition, at least the attitude and/or the motion of the portabledisplay apparatus that are obtained by rotating the portable displayapparatus about the direction of gravity in real space may becalculated. On the basis of the attitude and/or the motion of theportable display apparatus that are obtained by rotating the portabledisplay apparatus about the direction of gravity in real space, at leastone of the direction, the attitude, and the position of the playerobject may be controlled such that the player object rotates about thedirection of gravity set in the virtual world.

Based on the above, it is possible to control at least one of thedirection, the attitude, and the position of the player object so as todirect the player object leftward and rightward in the virtual world bydirecting the portable display apparatus leftward and rightward in realspace.

In addition, at least the attitude and/or the motion of the portabledisplay apparatus that are obtained by swinging the portable displayapparatus upward and downward about a horizontal direction perpendicularto the direction of gravity in real space may be calculated. On thebasis of the attitude and/or the motion of the portable displayapparatus that are obtained by swinging the portable display apparatusupward and downward about the horizontal direction in real space, atleast one of the direction, the attitude, and the position of the playerobject may be controlled such that the player object rotates about ahorizontal direction set in the virtual world.

Based on the above, it is possible to control at least one of thedirection, the attitude, and the position of the player object so as todirect the player object upward and downward in the virtual world bydirecting the portable display apparatus upward and downward in realspace.

In addition, at least the attitude and/or the motion of the portabledisplay apparatus that are obtained by rotating the portable displayapparatus about two axes orthogonal to a perspective direction of, andperpendicular to, the display screen of the portable display apparatusmay be calculated. At least one of the direction, the attitude, and theposition of the player object may be controlled such that in accordancewith the attitude and/or the motion of the portable display apparatusthat are obtained by rotating the portable display apparatus about thetwo axes, the player object rotates about two axes that correspond tothe two axes in real space and are included in the virtual world.

Based on the above, it is possible to control at least one of thedirection, the attitude, and the position of the player object so as todirect the player object upward, downward, leftward, and rightward inthe virtual world by moving the portable display apparatus so as torotate about two axes orthogonal to the perspective direction of adisplay screen of the portable display apparatus in real space.

In addition, at least the attitude and/or the motion of the portabledisplay apparatus that are obtained by rotating the portable displayapparatus about an axis along a width direction of the display screenand an axis along a height direction of the display screen may becalculated, each axis being orthogonal to the perspective direction. Atleast one of the direction, the attitude, and the position of the playerobject may be controlled such that: in accordance with the attitudeand/or the motion of the portable display apparatus that are obtained byrotating the portable display apparatus about the axis along the widthdirection, the player object rotates about a horizontal axis in thevirtual world; and in accordance with the attitude and/or the motion ofthe portable display apparatus that are obtained by rotating theportable display apparatus about the axis along the height direction,the player object rotates about a vertical axis in the virtual world.

Based on the above, it is possible to control at least one of thedirection, the attitude, and the position of the player object so as todirect the player object upward, downward, leftward, and rightward inthe virtual world by moving the portable display apparatus so as torotate about the height direction and the width direction of the displayscreen of the portable display apparatus in real space.

In addition, the game program may further cause the computer to executesetting a first virtual camera for generating an image of the virtualworld viewed from the player object or an image of the virtual worldincluding at least the player object, and controlling an attitude and/ora position of the first virtual camera on the basis of the calculatedattitude and/or motion of the portable display apparatus. In this case,the first image may be generated by superimposing the attack effectimage on the image of the virtual world viewed from the first virtualcamera.

Based on the above, the attitude and/or the motion of the first virtualcamera for generating an image of the virtual world to be displayed onthe portable display apparatus are controlled on the basis of theattitude and the motion of the portable display apparatus. This makes itpossible, for example, in accordance with the user directing theportable display apparatus in the direction that they wish to view, toprovide the user with an image as if peeping at the virtual worldthrough the portable display apparatus, and provide the user with afeeling as if being in the virtual world.

In addition, a direction of a line of sight of the first virtual cameramay be controlled so as to be the same as the controlled direction ofthe player object.

Based on the above, in accordance with a change in the direction of theplayer object, also the direction of the line of sight of the firstvirtual camera changes. As a result, the position and the attitude ofthe first virtual camera and the direction of the player object arecontrolled on the basis of the attitude and/or the motion of theportable display apparatus. This makes it possible, in accordance withthe user directing the portable display apparatus in the direction thatthey wish to view, to change the direction of the player object, andprovide the user with an image as if peeping at the virtual worldthrough the portable display apparatus. It is also possible to providethe user with a feeling as if being in the virtual world. Further, thedirection of the player object can be set on the basis of the attitudeof the portable display apparatus, and the virtual world viewed in thedirection of the player object is displayed on the portable displayapparatus. This achieves the operation of setting the direction of theplayer object in an intuitive manner. This facilitates setting thedirection of the player object to the direction desired by the user.

In addition, the portable display apparatus may include at least one ofa gyro sensor and an acceleration sensor. In this case, the attitudeand/or the motion of the portable display apparatus may be calculated onthe basis of data output from the at least one of the gyro sensor andthe acceleration sensor.

Based on the above, using the data that is output from the gyro sensorand indicates the angular velocity generated in the portable displayapparatus and/or the data that is output from the acceleration sensorand indicates the acceleration generated in the portable displayapparatus, it is possible to accurately calculate the attitude and themotion of the portable display apparatus.

In addition, an image that hinders a field of view toward the virtualworld when superimposed on the image of the virtual world may begenerated as the attack effect image. When the image is superimposed onthe image of the virtual world, the area in the attack effect image thatoverlaps the predetermined range may be repaired so that the field ofview toward the virtual world is not hindered.

Based on the above, when a specific attack has been received, the fieldof view toward the virtual world from the display apparatus of theportable display apparatus is hindered. This worsens the conditions inwhich the user plays the game, and therefore increases the level ofdifficulty of the game. A touch operation may be performed on the partwhere the field of view is hindered, whereby the effect of the attack isrepaired in the periphery of the touch position. This enables theoperation of repairing the effect of the attack in an intuitive manner.This makes it possible to improve virtual reality.

In addition, image data indicating the first image may be output to theportable display apparatus. The portable display apparatus may includean image data acquisition unit. The image data acquisition unit acquiresthe image data output from the game apparatus. In this case, a displayscreen of the portable display apparatus may display the first imageindicated by the image data acquired by the image data acquisition unit.

Based on the above, the portable display apparatus can function as aso-called thin-client terminal, which does not perform informationprocessing such as game processing.

In addition, the game program may further cause the computer to executegenerating compression image data by compressing the image dataindicating the first image. In this case, the generated compressionimage data may be output to the portable display apparatus. The imagedata acquisition unit may acquire the compression image data output fromthe game apparatus. The portable display apparatus may further include adisplay image decompression unit. The display image decompression unitdecompresses the compression image data to obtain the image dataindicating the first image. In this case, the display screen of theportable display apparatus may display the first image indicated by theimage data that has been acquired by the image data acquisition unit andhas been decompressed by the display image decompression unit.

Based on the above, the first image is decompressed and then output fromthe game apparatus to the portable display apparatus. This makes itpossible to output the first image at a high speed, and reduce delaycaused between the generation of the first image and the display of thefirst image on the portable display apparatus.

In addition, besides the first image, a second image representing thevirtual world may be further displayed on another display apparatusconnected to the game apparatus.

It should be noted that said another display apparatus described aboveis a display apparatus connected to the game apparatus, like a monitor 2according to the embodiment described later. Said another displayapparatus may be a component separate from the portable displayapparatus, and may be any apparatus so long as it is capable ofdisplaying the second image generated by the game apparatus. Forexample, said another display apparatus described above may beintegrated with the game apparatus (in a single housing).

Based on the above, when processing based on the operation of moving andchanging the attitude of the portable display apparatus is performed, itis possible to display the results of the processing not only on theportable display apparatus but also on said another display apparatusconnected to the game apparatus. This enables the user to use, inaccordance with the state of the operation or the user's preference,either one of images displayed on, for example, two apparatuses, andalso view an image suitable for an operation of the user. Further, it ispossible to use an image displayed on said another display apparatusconnected to the game apparatus, as, for example, an image to be viewedby another person different from the user. This makes it possible toprovide a viewing environment suitable also for the case where aplurality of people view the results of the processing.

In addition, the game program may further cause the computer to executegenerating compression image data by compressing the image dataindicating the first image. In this case, the generated compressionimage data may be output to the portable display apparatus, and, besidesthe compression image data, image data indicating the second image maybe output to said another display apparatus without being compressed.The portable display apparatus may include an image data acquisitionunit and a display image decompression unit. The image data acquisitionunit acquires the compression image data output from the game apparatus.The display image decompression unit decompresses the compression imagedata to obtain the image data indicating the first image. In this case,a display screen of the portable display apparatus may display the firstimage indicated by the image data that has been acquired by the imagedata acquisition unit and has been decompressed by the display imagedecompression unit.

Based on the above, the first image is decompressed and then output fromthe game apparatus to the portable display apparatus. This makes itpossible to output the first image at a high speed, and reduce delaycaused between the generation of the first image and the display of thefirst image on the portable display apparatus.

In addition, an image including the player object in the virtual worldviewed from a point of view different from a point of view toward thevirtual world for generating the first image may be displayed as thesecond image on said another display apparatus.

Based on the above, the same virtual world is displayed not only on theportable display apparatus but also on said another display apparatus,and images of the virtual world that are different in the point of vieware displayed thereon. This enables the user to use, in accordance withthe state of the operation or the user's preference, either one of theimages displayed on the two apparatuses when performing an operation.

In addition, a point of view toward the virtual world for generating thesecond image may be set at a position further away from the playerobject than a point of view toward the virtual world for generating thefirst image is from the player object. A range wider than a range of thevirtual world represented by the first image may be displayed as thesecond image on said another display apparatus.

Based on the above, an image of the virtual world in a display rangewider than that of an image of the virtual world displayed on theportable display apparatus is displayed on said another displayapparatus connected to a game apparatus. This makes it possible todisplay on each display apparatus, for example, an image suitable for anoperation of the user when the state of the virtual world is presentedto the user.

In addition, a point of view for generating the second image may be setat a position of viewing from a bird's-eye view the player object in thevirtual world. An image obtained by viewing from a bird's-eye view theplayer object placed in the virtual world may be displayed as the secondimage on said another display apparatus.

Based on the above, the same virtual world is displayed not only on theportable display apparatus but also on said another display apparatus,and an image of the virtual world obtained by looking down upon it isdisplayed on said another display apparatus. This makes it possible todisplay on each display apparatus, for example, an image suitable for anoperation of the user when the state of the virtual world is presentedto the user.

In addition, the exemplary embodiment may be carried out in the forms ofa game apparatus and a game system, each including units that performthe above processes, and a game processing method including the aboveoperations.

The exemplary embodiment makes it possible to improve virtual reality bycausing a player object in a virtual world to take action in accordancewith an operation of a user.

These and other objects, features, aspects and advantages of theexemplary embodiment will become more apparent from the followingdetailed description when taken in conjunction with the accompanyingdrawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is an external view showing an example of a game system 1according to a non-limiting exemplary embodiment;

FIG. 2 is a functional block diagram showing a non-limiting example of agame apparatus body 5 of FIG. 1;

FIG. 3 is a diagram showing a non-limiting example of the externalconfiguration of a terminal apparatus 6 of FIG. 1;

FIG. 4 is a diagram showing a non-limiting example of the state where auser holds the terminal apparatus 6;

FIG. 5 is a block diagram showing a non-limiting example of the internalconfiguration of the terminal apparatus 6 of FIG. 3;

FIG. 6 is a perspective view showing a non-limiting example of theappearance of a board-type controller 9 of FIG. 1;

FIG. 7 is a diagram showing a non-limiting example of a cross-sectionalview of the board-type controller 9 shown in FIG. 6 taken along lineA-A, and a non-limiting example of an enlarged view of a corner portionwhere a load sensor 94 is arranged;

FIG. 8 is a block diagram showing a non-limiting example of theelectrical configuration of the board-type controller 9 of FIG. 6;

FIG. 9 is a diagram showing a non-limiting example of the state of auser performing an operation using the terminal apparatus 6 and theboard-type controller 9;

FIG. 10A is a diagram showing an example of an image displayed on an LCD61 of the terminal apparatus 6;

FIG. 10B is a diagram showing an example of an image displayed on amonitor 2;

FIG. 11 is a diagram showing a non-limiting example where a terminalapparatus 6 has been rotated to the left and right, and a non-limitingexample of an image displayed on the LCD 61;

FIG. 12 is a diagram illustrating a non-limiting example of: therelationship between a terminal apparatus perspective directionprojected onto a horizontal plane in real space and an operationindication direction projected onto a horizontal plane in a virtualworld; and a player object Po controlled so as to be directed in adirection based on the operation indication direction;

FIG. 13 is a diagram illustrating a non-limiting example of theoperation indication direction obtained by rotating the terminalapparatus 6 to the left and right, and the player object Po controlledso as to be directed in a direction based on the operation indicationdirection;

FIG. 14A is a diagram illustrating a non-limiting example of a barrelleft-right operation range and a virtual camera left-right operationrange;

FIG. 14B is a diagram illustrating a non-limiting example of a barrelup-down operation range and a virtual camera up-down operation range;

FIG. 15A is a diagram showing a non-limiting example of a dirt image inwhich dirt clumps Bd are represented so as to be attached to the playerobject Po;

FIG. 15B is a diagram showing a non-limiting example of an image inwhich areas in the dirt clumps Bd represented so as to be attached tothe player object Po are removed by a touch operation;

FIG. 16 is a diagram showing a non-limiting example of data and programsthat are stored in a main memory of the game apparatus body 5 of FIG. 1;

FIG. 17 is a flow chart showing a non-limiting example of gameprocessing performed by the game apparatus body 5 of FIG. 1;

FIG. 18 is a subroutine flow chart showing a non-limiting example of agame control process in step 44 in FIG. 17;

FIG. 19 is a subroutine flow chart showing a non-limiting example of aplayer object setting process in step 83 in FIG. 18;

FIG. 20 is a subroutine flow chart showing a non-limiting example of anoperation indication direction calculation process in step 121 in FIG.19; and

FIG. 21 is a subroutine flow chart showing a non-limiting example of adischarge object setting process in step 130 of FIG. 19;

FIG. 22 is a subroutine flow chart showing a non-limiting example of anattack reception operation in step 131 of FIG. 19; and

FIG. 23 is a diagram illustrating a non-limiting example of movementvectors Vw1 through Vw15 respectively set for discharge objects W1through W15 that move in the virtual world.

DETAILED DESCRIPTION OF NON-LIMITING EXAMPLE EMBODIMENTS

With reference to FIG. 1, a game apparatus for executing a game programaccording to an exemplary embodiment and a game system including thegame apparatus is described. Hereinafter, in order to provide a specificdescription, a stationary game apparatus body 5 is used as an example ofthe game apparatus, and a game system including the game apparatus body5 is described. FIG. 1 is an external view showing an example of thegame system 1 including the stationary game apparatus body 5. FIG. 2 isa block diagram showing an example of the game apparatus body 5.Hereinafter, the game system 1 is described.

As shown in FIG. 1, the game system 1 includes a household televisionreceiver (hereinafter referred to as a “monitor”) 2 which is an exampleof display means, and the stationary game apparatus 3 connected to themonitor 2 via a connection cord. The monitor 2 includes loudspeakers 2 afor outputting, in the form of sound, a sound signal outputted from thegame apparatus 3. Further, the game apparatus 3 includes: an opticaldisk 4 having stored therein a program, which is an example of the gameprogram according to the exemplary embodiment; the game apparatus body 5having a computer for executing the program stored in the optical disk 4to display a game screen on the monitor 2; a terminal apparatus 6; acontroller 7 for providing the game apparatus body 5 with operationinformation used to operate, for example, objects displayed on thedisplay screen; and a board-type controller 9. The game system 1performs game processing on the game apparatus body 5 in accordance witha game operation using at least one of the terminal apparatus 6, thecontroller 7, and the board-type controller 9, and displays a game imageobtained by the game processing on the monitor 2 and/or the terminalapparatus 6. The game apparatus body 5 is wirelessly connected to theterminal apparatus 6, the controller 7, and the board-type controller 9so as to enable wireless communication therebetween. For example, thewireless communication is performed according to the Bluetooth(registered trademark) standard or the IEEE 802.11n standard. Thewireless communication, however, may be performed in accordance withother standards such as standards for infrared communication.

The optical disk 4, typifying an information storage medium used for thegame apparatus body 5 in an exchangeable manner, is detachably insertedin the game apparatus body 5. The optical disk 4 has stored therein thegame program to be performed by the game apparatus body 5. The gameapparatus body 5 has, on a front surface thereof, an insertion openingfor the optical disk 4. The game apparatus body 5 reads and executes thegame program stored in the optical disk 4 inserted into the insertionopening to perform the game processing.

The monitor 2 is connected to the game apparatus body 5 via a connectioncord. The monitor 2 displays a game image obtained by the gameprocessing performed by the game apparatus body 5. The monitor 2includes the loudspeakers 2 a. The loudspeakers 2 a each output a gamesound obtained as a result of the game processing. In anotherembodiment, the game apparatus body 5 and a stationary display apparatusmay be integrated with each other. The communication between the gameapparatus body 5 and the monitor 2 may be wireless communication.

The game apparatus body 5 has mounted thereto a flash memory 17 (seeFIG. 2) which functions as a backup memory for fixedly storing data suchas saved data. The game apparatus body 5 executes the game program orthe like stored in the optical disk 4, and displays a result thereof asa game image on the monitor 2 and/or the terminal apparatus 6. The gameprogram or the like to be executed may be stored in advance in the flashmemory 17 as well as in the optical disk 4. Further, the game apparatusbody 5 may reproduce a state of a game played in the past, using thesaved data stored in the flash memory 17, and display an image of thegame state on the monitor 2 and/or the terminal apparatus 6. A user ofthe game apparatus 3 can enjoy the game progress by operating at leastone of the terminal apparatus 6, the controller 7, and the board-typecontroller 9 while viewing the game image displayed on the monitor 2and/or the terminal apparatus 6.

The controller 7 and the board-type controller 9 each wirelesslytransmit transmission data such as operation information, using, forexample, the Bluetooth technology, to the game apparatus body 5 having acontroller communication module 19. The controller 7 is operation meansfor performing, for example, selection of options displayed on thedisplay screen of the monitor 2. The controller 7 includes a housingwhich is small enough to be held by one hand, and a plurality ofoperation buttons (including a cross key and the like) which are exposedat the surface of the housing. In addition, as is described later, thecontroller 7 includes an imaging information calculation section fortaking an image viewed from the controller 7. As exemplary imagingtargets of the imaging information calculation section, two LED modules(hereinafter referred to as “markers”) 8L and 8R are provided in thevicinity of the display screen of the monitor 2 (above the screen inFIG. 1). Although details will be described later, a user (player) isallowed to perform a game operation while moving the controller 7, andthe game apparatus body 5 uses a marker 8 to calculate the movement,position, attitude and the like of the controller 7. The marker 8 hastwo markers 8L and 8R at both ends thereof. Specifically, the marker 8L(as well as the marker 8R) includes one or more infrared LEDs (LightEmitting Diodes), and emits infrared light forward from the monitor 2.The marker 8 is connected to the game apparatus body 5, so that the gameapparatus body 5 can control the infrared LEDs included in the marker 8to be lit on or off. The marker 8 is a portable unit, so that the useris allowed to place the marker 8 in a given position. Although FIG. 1shows a case where the marker 8 is placed on the monitor 2, the locationand direction of the marker 8 may be appropriately selected. Further,the controller 7 is capable of receiving, at a communication section,transmission data wirelessly transmitted from the controllercommunication module 19 of the game apparatus body 5, to generate asound or vibration based on the transmission data.

In another embodiment, the controller 7 and/or the board-type controller9 may be wire-connected to the game apparatus body 5. Further, in theexemplary embodiment, the game system 1 includes a controller 7 and aboard-type controller 9. The game apparatus body 5, however, is capableof communicating with a plurality of controllers 7 and a plurality ofboard-type controllers 9. Therefore, a plurality of players can play agame using a predetermined number of controllers 7 and board-typecontroller 9 simultaneously.

The controller 7 includes a housing which is formed by, for example,plastic molding, and has a plurality of operation sections (operationbuttons) in the housing 71. Then, the controller 7 transmits, to thegame apparatus body 5, operation data indicating the states of inputsprovided to the operation sections (indicating whether or not eachoperation button has been pressed).

In addition, the controller 7 has the imaging information calculationsection that analyzes image data of an image captured by capturing meansand determines an area having a high brightness, and thereby calculatesthe position of the center of gravity, the size, and the like of thearea. For example, the imaging information calculation section hascapturing means fixed in the housing of the controller 7, and uses as animaging target a marker that outputs infrared light, such as a markersection 65 of the terminal apparatus 6 and/or the marker 8. The imaginginformation calculation section calculates the position of the imagingtarget in a captured image captured by the capturing means, andtransmits, to the game apparatus body 5, marker coordinate dataindicating the calculated position. The marker coordinate data variesdepending on the direction (the angle of tilt) or the position of thecontroller 7, and therefore, the game apparatus body 5 can calculate thedirection and the position of the controller 7 using the markercoordinate data.

In addition, the controller 7 includes therein an acceleration sensorand/or a gyro sensor. The acceleration sensor detects the accelerationgenerated in the controller 7 (including the gravitationalacceleration), and transmits, to the game apparatus body 5, dataindicating the detected acceleration. The acceleration detected by theacceleration sensor varies depending on the direction (the angle oftilt) or the movement of the controller 7, and therefore, the gameapparatus body 5 can calculate the direction and the movement of thecontroller 7 using the acquired acceleration data. The gyro sensordetects the angular velocities generated about three axes set in thecontroller 7, and transmits, to the game apparatus body 5, angularvelocity data indicating the detected angular velocities. Theacceleration detected by the gyro sensor varies depending on thedirection (the angle of tilt) or the movement of the controller 7, andtherefore, the game apparatus body 5 can calculate the direction and themovement of the controller 7 using the acquired acceleration data. Asdescribed above, the user is allowed to perform a game operation bypressing any of the operation sections 72 provided on the controller 7,and moving the controller 7 so as to change the position and theattitude (tilt) thereof.

The controller 7 has a loudspeaker and a vibrator. The controller 7processes sound data transmitted from the game apparatus body 5, andoutputs sound corresponding to the sound data from the loudspeaker.Further, the controller 7 processes vibration data transmitted from thegame apparatus body 5, and generates vibration by actuating the vibratorin accordance with the vibration data. It should be noted that in theexemplary embodiment described later, it is possible to play a gamewithout using the controller 7. A detailed configuration of theboard-type controller 9 will be described later.

The terminal apparatus 6 is a portable apparatus that is small enough tobe held by the user, and the user is allowed to move the terminalapparatus 6 with hands, or place the terminal apparatus 6 at anylocation. Although a detailed configuration of the terminal apparatus 6will be described later, the terminal apparatus 6 includes an LCD(Liquid Crystal Display) 61 as display means, and input means (a touchpanel 62, a gyro sensor 604, and the like described later). The terminalapparatus 6 and the game apparatus body 5 (a terminal communicationmodule 28 (see FIG. 2)) are capable of communicating with each otherwirelessly or wired. The terminal apparatus 6 receives, from the gameapparatus body 5, data of an image (e.g., a game image) generated in thegame apparatus body 5, and displays the image represented by the data onan LCD 61. Although the LCD 61 is used as a display apparatus in theexemplary embodiment, the terminal apparatus 6 may include a given otherdisplay apparatus, such as a display apparatus utilizing EL (ElectroLuminescence), for example. Further, the terminal apparatus 6 transmits,to the game apparatus body 5 having the terminal communication module28, operation data representing the content of an operation performed onthe terminal apparatus 6.

Next, with reference to FIG. 2, the internal configuration of the gameapparatus body 5 is described. FIG. 2 is a block diagram showing anexample of the internal configuration of the game apparatus body 5. Thegame apparatus body 5 includes a CPU (Central Processing Unit) 10, asystem LSI (Large Scale Integration) 11, an external main memory 12, aROM/RTC (Read Only Memory/Real Time Clock) 13, a disk drive 14, an AV-IC(Audio Video-Integrated Circuit) 15 and the like.

The CPU 10, serving as a game processor, executes a program stored inthe optical disk 4 to perform a process. The CPU 10 is connected to thesystem LSI 11. In addition to the CPU 10, the external main memory 12,the ROM/RTC 13, the disk drive 14, and the AV-IC 15 are connected to thesystem LSI 11. The system LSI 11 performs processes such as control ofdata transmission between the respective components connected thereto,generation of an image to be displayed, and acquisition of data from anexternal apparatus. The internal configuration of the system LSI 11 willbe described later. The external main memory 12, which is a volatilememory, stores programs loaded from the optical disk 4 or the flashmemory 17, and stores various data. The external main memory 12 is usedas a work area and a buffer area for the CPU 10. The ROM/RTC 13 includesa ROM (so-called boot ROM) incorporating a program for booting the gameapparatus body 5, and a clock circuit (RTC) for counting time. The diskdrive 14 reads, from the optical disk 4, program data, texture data andthe like, and writes the read data into an internal main memory 35described below or the external main memory 12.

The system LSI 11 includes an input/output processor (I/O processor) 31,a GPU (Graphics Processor Unit) 32, a DSP (Digital Signal Processor) 33,a VRAM (Video RAM) 34, and the internal main memory 35. These components31 to 35 are connected to each other via an internal bus (not shown).

The GPU 32, which is a part of rendering means, generates an image inaccordance with a graphics command (draw command) supplied from the CPU10. The VRAM 34 stores data (such as polygon data and texture data) usedby the GPU 32 to execute the graphics command. When an image isgenerated, the GPU 32 generates image data using the data stored in theVRAM 3. In the exemplary embodiment, the game apparatus body 5 maygenerate both a game image to be displayed on the monitor 2 and a gameimage to be displayed on the terminal apparatus 6. Hereinafter, the gameimage to be displayed on the monitor 2 may be referred to as a “monitorgame image”, and the game image to be displayed on the terminalapparatus 6 may be referred to as a “terminal game image”.

The DSP 33, serving as an audio processor, generates sound data usingsound data and sound waveform (tone quality) data stored in the internalmain memory 35 and the external main memory 12. In the exemplaryembodiment, similarly to the game images, both a game sound to be outputfrom the loudspeakers 2 a of the monitor 2 and a game sound to be outputfrom the loudspeakers of the terminal apparatus 6 may be generated.Hereinafter, the game sound to be output from the monitor 2 may bereferred to as a “monitor game sound”, and the game sound to be outputfrom the terminal apparatus 6 may be referred to as a “terminal gamesound”.

Among the image data and sound data generated by the game apparatus body5, the image data and sound data to be output to the monitor 2 are readby the AV-IC 15. The AV-IC 15 outputs the read image data to the monitor2 via an AV connector 16, and outputs the read sound data to theloudspeakers 2 a included in the monitor 2. Thereby, an image isdisplayed on the monitor 2, and a sound is output from the loudspeakers2 a.

Further, among the image data and sound data generated by the gameapparatus body 5, the image data and sound data to be output to theterminal apparatus 6 are transmitted to the terminal apparatus 6 by theI/O processor 31 or the like. Data transmission to the terminalapparatus 6 by the I/O processor 31 or the like will be described later.

The I/O processor 31 performs data reception and transmission with thecomponents connected thereto, and download of data from an externalapparatus. The I/O processor 31 is connected to the flash memory 17, thenetwork communication module 18, the controller communication module 19,an extension connector 20, a memory card connector 21, and a codec LSI27. An antenna 23 is connected to the controller communication module19. The codec LSI 27 is connected to the terminal communication module28, and an antenna 29 is connected to the terminal communication module28.

The game apparatus body 5 is connected to a network such as the Internetso as to communicate with external information processing apparatuses(for example, other game apparatuses or various servers). That is, theI/O processor 31 is connected to a network via the network communicationmodule 18 and the antenna 22 so as to communicate with externalinformation processing apparatuses connected to the network. The I/Oprocessor 31 accesses the flash memory 17 at regular intervals so as todetect for data to be transmitted to the network. When data to betransmitted is detected, the data is transmitted to the network via thenetwork communication module 18 and the antenna 22. Further, the I/Oprocessor 31 receives, via the network, the antenna 22 and the networkcommunication module 18, data transmitted from the external informationprocessing apparatuses or data downloaded from a download server, andstores the received data in the flash memory 17. The CPU 10 executes aprogram, and reads the data stored in the flash memory 17 to use thedata for execution of the program. The flash memory 17 may store notonly the data transmitted and received between the game apparatus body 5and the external information processing apparatuses, but also saved data(result data or progress data of the process) of the game played withthe game apparatus body 5. Further, the flash memory 17 may storeprograms such as a game program.

The game apparatus body 5 can receive operation data from the controller7 and/or the board-type controller 9. That is, the I/O processor 31receives, via the antenna 23 and the controller communication module 19,operation data or the like transmitted from the controller 7 and/or theboard-type controller 9, and stores (temporarily) the data in a bufferregion of the internal main memory 35 or the external main memory 12.Similarly to the external main memory 12, the internal main memory 35may store a program loaded from the optical disk 4 or a program loadedfrom the flash memory 17, and various data. The internal main memory 35may be used as a work region or buffer region of the CPU 10.

The game apparatus body 5 is capable of transmitting/receiving imagedata, sound data and the like to/from the terminal apparatus 6. Whentransmitting a game image (terminal game image) to the terminalapparatus 6, the I/O processor 31 outputs data of a game image generatedby the GPU 32 to the codec LSI 27. The codec LSI 27 performs apredetermined compression process on the image data supplied from theI/O processor 31. The terminal communication module 28 performs wirelesscommunication with the terminal apparatus 6. Accordingly, the image datacompressed by the codec LSI 27 is transmitted by the terminalcommunication module 28 to the terminal apparatus 6 via the antenna 29.In the exemplary embodiment, the codec LSI 27 compresses the image datausing a highly efficient compression technique, for example, the H.264standard. The codec LSI 27 may adopt other compression techniques. Whenthe communication rate is sufficiently high, uncompressed image data maybe transmitted. The terminal communication module 28 is, for example, aWi-Fi certified communication module. The terminal communication module28 may perform wireless communication with the terminal apparatus 6 at ahigh speed using, for example, the technique of MIMO (Multiple InputMultiple Output) adopted in the IEEE 802.11n standard, or may use othercommunication techniques.

The game apparatus body 5 transmits, to the terminal apparatus 6, sounddata as well as the image data. That is, the I/O processor 31 outputssound data generated by the DSP 33 to the terminal communication module28 via the codec LSI 27. The codec LSI 27 performs a compression processon the sound data in a similar manner to that for the image data. Anycompression technique may be adopted for the sound data. In anotherembodiment, uncompressed sound data may be transmitted. The terminalcommunication module 28 transmits the compressed image data and sounddata to the terminal apparatus 6 via the antenna 29.

The game apparatus body 5 transmits, in addition to the image data andsound data, various control data to the terminal apparatus 6, wherenecessary. The control data represent control instructions for thecomponents included in the terminal apparatus 6, such as an instructionto control on/off of a marker section (a marker section 65 shown in FIG.5), and an instruction to control image taking of a camera (a camera 66shown in FIG. 10). The I/O processor 31 transmits the control data tothe terminal apparatus 6 in response to an instruction from the CPU 5.In the exemplary embodiment, the codec LSI 27 does not perform a datacompression process on the control data. Alternatively, in anotherembodiment, the codec LSI 27 may perform a compression process on thecontrol data. The above data transmitted from the game apparatus body 5to the terminal apparatus 6 may be encrypted where necessary, or may notbe encrypted.

The game apparatus body 5 can receive various data from the terminalapparatus 6. Although details will be described later, in the exemplaryembodiment, the terminal apparatus 6 transmits operation data, imagedata, and sound data. The respective data transmitted from the terminalapparatus 6 are received by the terminal communication module 28 via theantenna 29. The image data and sound data transmitted from the terminalapparatus 6 have been subjected to a similar compression process to thatfor the image data and sound data transmitted from the game apparatusbody 5 to the terminal apparatus 6. Accordingly, these image data andsound data are transmitted from the terminal communication module 28 tothe codec LSI 27, and subjected to a decompression process by the codecLSI 27. The decompressed data are output to the I/O processor 31. On theother hand, the operation data transmitted from the terminal apparatus 6is smaller in amount than the image data and sound data, and therefore,the operation data does not need to be compressed. The operation datamay be encrypted where necessary, or may not be encrypted. Accordingly,the operation data, which has been received by the terminalcommunication module 28, is output to the I/O processor 31 via the codecLSI 27. The I/O processor 31 stores (temporarily) the data received fromthe terminal apparatus 6 in the buffer region of the internal mainmemory 35 or the external main memory 12.

The game apparatus body 5 is connectable to other devices and externalstorage media. That is, an extension connector 20 and a memory cardconnector 21 are connected to the I/O processor 31. The expansionconnector 20 is an interface connector as typified by a USB and an SCSI,and is capable of performing communication with the network, instead ofthe network communication module 18, by connecting thereto a medium suchas an external storage medium, a peripheral device such as anothercontroller, or a wired communication connector. The memory cardconnector 21 is a connector for connecting thereto an external storagemedium such as a memory card. For example, the I/O processor 31 accessesthe external storage medium via the expansion connector 20 or the memorycard connector 21 to save or read data.

The game apparatus body 5 includes (on the front main surface thereof,for example) a power button 24, a reset button 25, an insertion slot inwhich the optical disk 4 is inserted, an eject button 26 for ejectingthe optical disk 4 from the insertion slot of the game apparatus body 5,and the like. The power button 24 and the reset button 25 are connectedto the system LSI 11. When the power button 24 is turned on, therespective components of the game apparatus body 5 are supplied withpower. When the reset button 25 is pressed, the system LSI 11re-executes the boot program of the game apparatus body 5. The ejectbutton 26 is connected to the disk drive 14. When the eject button 26 ispressed, the optical disk 4 is ejected from the disk drive 14.

In another embodiment, some of the components of the game apparatus body5 may be constituted as an extension device separated from the gameapparatus body 5. At this time, the extension device may be connected tothe game apparatus body 5 via the extension connector 20. Specifically,the extension device may include, for example, the codec LSI 27, theterminal communication module 28, and the antenna 29, and may bedetachably connected to the extension connector 20. Thus, by connectingthe extension device to the game apparatus body which does not have theabove components, the game apparatus body can be made capable ofcommunicating with the terminal apparatus 6.

Next, with reference to FIGS. 3 through 5, the configuration of theterminal apparatus 6 is described. FIG. 3 is a diagram showing anexample of the external configuration of the terminal apparatus 6. Morespecifically, (a) of FIG. 3 is a front view of the terminal apparatus 6,(b) of FIG. 3 is a top view, (c) of FIG. 3 is a right side view, and (d)of FIG. 3 is a bottom view. FIG. 4 shows an example of the state where auser holds the terminal apparatus 6 with both hands.

As shown in FIG. 3, the terminal apparatus 6 includes a housing 60 whichgenerally has a horizontally long plate-like rectangular shape. Thehousing 60 is small enough to be held by the user. Therefore, the useris allowed to move the terminal apparatus 6 with hands, and change thelocation of the terminal apparatus 6.

The terminal apparatus 6 includes an LCD 61 on a front surface of thehousing 60. The LCD 61 is provided near the center of the front surfaceof the housing 60. Therefore, as shown in FIG. 4, the user, holding thehousing 60 at portions to the left and right of the LCD 61, is allowedto move the terminal apparatus 6 while viewing a screen of the LCD 61.FIG. 4 shows an example where the user holds the terminal apparatus 6horizontally (i.e., with the longer sides of the terminal apparatus 6being oriented horizontally) by holding the housing 60 at portions tothe left and right of the LCD 61. The user, however, may hold theterminal apparatus 6 vertically (i.e., with the longer sides of theterminal apparatus 6 being oriented vertically).

As shown in (a) of FIG. 3, the terminal apparatus 6 includes, asoperation means, a touch panel 62 on the screen of the LCD 61. In theexemplary embodiment, the touch panel 62 is, but is not limited to, aresistive film type touch panel. However, a touch panel of a given type,such as electrostatic capacitance type, may be used. The touch panel 62may be of single touch type or multiple touch type. In the exemplaryembodiment, the touch panel 62 has the same resolution (detectionaccuracy) as that of the LCD 61. The resolution of the touch panel 62and the resolution of the LCD 61, however, do not need to be the same.Although an input to the touch panel 62 is usually performed using atouch pen, in addition to the touch pen, a finger of the user may beused to perform an input to the touch panel 62. The housing 60 may havean opening for accommodating the touch pen used to perform an operationto the touch panel 62. The terminal apparatus 6 has the touch panel 62,and therefore, the user is allowed to operate the touch panel 62 whilemoving the terminal apparatus 6. That is, the user is allowed todirectly (using the touch panel 62) perform an input to the screen ofthe LCD 61 while moving the LCD 61.

As shown in FIG. 3, the terminal apparatus 6 has, as operation means,two analog sticks 63A and 63B, and a plurality of operation buttons 64Athrough 64L. The analog sticks 63A and 63B are each a device fordesignating a direction. The analog sticks 63A and 63B are eachconfigured such that a stick part thereof to be operated by a finger ofthe user is slidable or tiltable in a given direction (at a given anglein a given direction such as the upward, the downward, the leftward, therightward, or the diagonal direction) with respect to the front surfaceof the housing 60. The left analog stick 63A is provided to the left ofthe screen of the LCD 61, and the right analog stick 63B is provided tothe right of the screen of the LCD 61. Therefore, the user is allowed toperform an input for designating a direction using the analog stick 63Aor 63B with either the left or right hand. Further, as shown in FIG. 4,the analog sticks 63A and 63B are positioned so as to be operated by theuser holding the left and right portions of the terminal apparatus 6.Therefore, the user is allowed to easily operate the analog sticks 63Aand 63B when the user holds and moves the terminal apparatus 6.

The operation buttons 64A through 64L are each operation means forperforming a predetermined input. As described below, the operationbuttons 64A through 64L are positioned so as to be operated by the userholding the left and right portions of the terminal apparatus 6 (seeFIG. 4). Accordingly, the user is allowed to easily operate theoperation means when the user holds and moves the terminal apparatus 6.

As shown in (a) of FIG. 3, among the operation buttons 64A through 64L,the cross button (direction input button) 64A and the operation buttons64B through 64H are provided on the front surface of the housing 60. Theoperation buttons 64A through 64H are positioned so as to be operated bya thumb of the user (see FIG. 4).

The cross button 64A is provided to the left of the LCD 61 and beneaththe left analog stick 63A. That is, the cross button 64A is positionedso as to be operated by the left hand of the user. The cross button 64Ais cross-shaped, and is capable of indicating an upward, a downward, aleftward, or a rightward direction. The operation buttons 64B through64D are provided beneath the LCD 61. The three operation buttons 64Bthrough 64D are positioned so as to be operated by the right and lefthands of the user. The four operation buttons 64E through 64H areprovided to the right of the LCD 61 and beneath the right analog stick63B. That is, the four operation buttons 64E through 64H are positionedso as to be operated by the right hand of the user. Further, the fouroperation buttons 64E through 64H are positioned upward, downward,leftward, and rightward, respectively, with respect to a center positionof the four operation buttons. Accordingly, the terminal apparatus 6 maycause the four operation buttons 64E through 64H to function as buttonswhich allow the user to designate an upward, a downward, a leftward, ora rightward direction.

As shown in (a), (b), and (c) of FIG. 3, a first L button 641 and afirst R button 64J are provided on diagonal upper portions (an upperleft portion and an upper right portion) of the housing 60.Specifically, the first L button 641 is provided on the left end of theupper side surface of the plate-shaped housing 60 so as to protrude fromthe upper and left side surfaces. The first R button 64J is provided onthe right end of the upper side surface of the housing 60 so as toprotrude from the upper and right side surfaces. In this way, the firstL button 641 is positioned so as to be operated by the index finger ofthe left hand of the user, and the first R button 64J is positioned soas to be operated by the index finger of the right hand of the user (seeFIG. 4).

As shown in (b) and (c) of FIG. 3, leg parts 68A and 68B are provided soas to protrude from a rear surface (i.e., a surface reverse of the frontsurface on which the LCD 61 is provided) of the plate-shaped housing 60,and a second L button 64K and a second R button 64L are provided so asto protrude from the leg parts 68A and 68B, respectively. Specifically,the second L button 64K is provided at a slightly upper position on theleft side (the left side as viewed from the front surface side) of therear surface of the housing 60, and the second R button 64L is providedat a slightly upper position on the right side (the right side as viewedfrom the front-surface side) of the rear surface of the housing 60. Inother words, the second L button 64K is provided at a positionsubstantially opposite to the left analog stick 63A provided on thefront surface, and the second R button 64L is provided at a positionsubstantially opposite to the right analog stick 63B provided on thefront surface. The second L button 64K is positioned so as to beoperated by the middle finger of the left hand of the user, and thesecond R button 64L is positioned so as to be operated by the middlefinger of the right hand of the user (see FIG. 4). Further, as shown in(c) of FIG. 3, the leg parts 68A and 68B each have a surface facingobliquely upward, and the second L button 64K and the second R button64L are provided on the oblique surfaces of the leg parts 68A and 68B,respectively. Thus, the second L button 64K and the second R button 64Lhave button surfaces facing obliquely upward. It is supposed that themiddle finger of the user moves vertically when the user holds theterminal apparatus 6, and therefore, the upward facing button surfacesallow the user to easily press the second L button 64K and the second Rbutton 64L. Further, the leg parts 68A and 68B provided on the rearsurface of the housing 60 allow the user to easily hold the housing 60.Moreover, the operation buttons provided on the leg parts 68A and 68Ballow the user to easily perform operation while holding the housing 60.

In the terminal apparatus 6 shown in FIG. 3, the second L button 64K andthe second R button 64L are provided on the rear surface of the housing60. Therefore, if the terminal apparatus 6 is placed with the screen ofthe LCD 61 (the front surface of the housing 60) facing upward, thescreen of the LCD 61 may not be perfectly horizontal. Accordingly, inanother embodiment, three or more leg parts may be provided on the rearsurface of the housing 60. In this case, if the terminal apparatus 6 isplaced on a floor with the screen of the LCD 61 facing upward, the threeor more leg parts contact the floor. Thus, the terminal apparatus 6 canbe placed with the screen of the LCD 61 being horizontal. Such ahorizontal placement of the terminal apparatus 6 may be achieved byproviding detachable leg parts on the rear surface of the housing 60.

The respective operation buttons 64A through 64L are assigned functions,where necessary, in accordance with a game program. For example, thecross button 64A may be used for direction designation operation,selection operation, and the like, and the operation buttons 64E through64H may be used for determination operation, cancellation operation, andthe like.

The terminal apparatus 6 includes a power button (not shown) for turningon/off the power of the terminal apparatus 6. The terminal apparatus 6may include an operation button for turning on/off screen display of theLCD 61, an operation button for performing connection setting (pairing)with the game apparatus body 5, and an operation button for adjustingthe volume of loudspeakers (loudspeakers 607 shown in FIG. 5).

As shown in (a) of FIG. 3, the terminal apparatus 6 includes a markersection (a marker section 65 shown in FIG. 5) including a marker 65A anda marker 65B, on the front surface of the housing 60. For example, themarker section 65 is provided above the LCD 61. The markers 65A and 65Bare each constituted by one or more infrared LEDs, like the markers 8Land 8R of the marker 8. The marker section 65 is used, like the marker8, for causing the game apparatus body 5 to calculate a movement or thelike of the controller 7 with respect to the marker section 65. The gameapparatus body 5 is capable of controlling the infrared LEDs of themarker section 65 to be on or off.

The terminal apparatus 6 includes a camera 66 as imaging means. Thecamera 66 includes an image pickup element (e.g., a CCD image sensor ora CMOS image sensor) having a predetermined resolution, and a lens. Forexample, the camera 66 is provided on the front surface of the housing60. Accordingly, the camera 66 is capable of taking an image of the faceof the user holding the terminal apparatus 6. For example, the camera 66is capable of taking an image of the user playing a game while viewingthe LCD 61.

The terminal apparatus 6 has a microphone (a microphone 609 shown inFIG. 5) as sound input means. A microphone hole 60 b is provided in thefront surface of the housing 60. The microphone 609 is embedded in thehousing 60 at a position inside the microphone hole 60 b. The microphone609 detects for a sound, such as user's voice, around the terminalapparatus 6.

The terminal apparatus 6 has loudspeakers (loudspeakers 607 shown inFIG. 5) as sound output means. As shown in (d) of FIG. 3, speaker holes60 a are provided in the lower side surface of the housing 60. A soundis output through the speaker holes 60 a from the loudspeakers 607. Inthe exemplary embodiment, the terminal apparatus 6 has two loudspeakers,and the speaker holes 60 a are provided at positions corresponding to aleft loudspeaker and a right loudspeaker.

The terminal apparatus 6 includes an extension connector 67 forconnecting another device to the terminal apparatus 6. In the exemplaryembodiment, as shown in (d) of FIG. 3, the extension connector 67 isprovided in the lower side surface of the housing 60. Any device may beconnected to the extension connection 67. For example, a controller (agun-shaped controller or the like) used for a specific game or an inputdevice such as a keyboard may be connected to the extension connector67. If another device does not need to be connected, the extensionconnector 67 does not need to be provided.

In the terminal apparatus 6 shown in FIG. 3, the shapes of the operationbuttons and the housing 60, the number of the respective components, andthe positions in which the components are provided are merely examples.The shapes, numbers, and positions may be different from those describedabove.

Next, with reference to FIG. 5, the internal configuration of theterminal apparatus 6 is described. FIG. 5 is a block diagram showing anexample of the internal configuration of the terminal apparatus 6. Asshown in FIG. 5, the terminal apparatus 6 includes, in addition to thecomponents shown in FIG. 3, a touch panel controller 601, a magneticsensor 602, a gyro sensor 604, a user interface controller (UIcontroller) 605, a codec LSI 606, loudspeakers 607, a sound IC 608, amicrophone 609, a wireless module 610, an antenna 611, an infraredcommunication module 612, a flash memory 613, a power supply IC 614, abattery 615, and a vibrator 619. These electronic components are mountedon an electronic circuit board and accommodated in the housing 60.

The UI controller 605 is a circuit for controlling data input to variousinput/output sections and data output from various input/outputsections. The UI controller 605 is connected to the touch panelcontroller 601, the analog stick 63 (the analog sticks 63A and 63B), theoperation button 64 (the operation buttons 64A through 64L), the markersection 65, the magnetic sensor 602, the acceleration sensor 603, thegyro sensor 604, and the vibrator 619. Further, the UI controller 605 isconnected to the codec LSI 606 and the extension connector 67. The powersupply IC 614 is connected to the UI controller 605, so that power issupplied to the respective components through the UI controller 605. Theinternal battery 615 is connected to the power supply IC 614, so thatpower is supplied from the battery 615. Further, a battery charger 616or a cable, which is supplied with power from an external power supply,may be connected to the power supply IC 614 via a connector or the like.In this case, the terminal apparatus 6 can be supplied with power andcharged from the external power supply using the battery charger 616 orthe cable. Charging of the terminal apparatus 6 may be performed bysetting the terminal apparatus 6 on a cradle (not shown) having acharging function.

The touch panel controller 601 is a circuit which is connected to thetouch panel 62 and controls the touch panel 62. The touch panelcontroller 601 generates a predetermined form of touch position data, onthe basis of a signal from the touch panel 62, and outputs the touchposition data to the UI controller 605. The touch position datarepresents coordinates of a position at which an input is performed onan input surface of the touch panel 62. The touch panel controller 601reads a signal from the touch panel 62 and generates touch position dataevery predetermined period of time. Further, various controlinstructions on the touch panel 62 are output from the UI controller 605to the touch panel controller 601.

The analog stick 63 outputs, to the UI controller 605, stick datarepresenting a direction in which the stick part operated by a finger ofthe user slides (or tilts), and the amount of the sliding (tilting). Theoperation button 64 outputs, to the UI controller 605, operation buttondata representing an input state of each of the operation buttons 64Athrough 64L (whether or not the operation button is pressed).

The magnetic sensor 602 detects the magnitude and direction of amagnetic field to detect an orientation. Orientation data representingthe detected orientation is output to the UI controller 605. The UIcontroller 605 outputs, to the magnetic sensor 602, a controlinstruction for the magnetic sensor 602. Examples of the magnetic sensor602 include: an MI (Magnetic Impedance) sensor, a fluxgate sensor, ahall sensor, a GMR (Giant Magneto Resistance) sensor, a TMR (TunnelingMagneto Resistance) sensor, and an AMR (Anisotropic Magneto Resistance)sensor. Any sensor, however, may be adopted as long as the sensor candetect an orientation. Strictly speaking, the obtained orientation datadoes not represent an orientation in a place where a magnetic field isgenerated in addition to the geomagnetism. Even in such a case, it ispossible to calculate a change in the attitude of the terminal apparatus6 because the orientation data changes when the terminal apparatus 6moves.

The acceleration sensor 603 is provided inside the housing 60. Theacceleration sensor 603 detects the magnitudes of linear accelerationsalong three axial directions (the x-axis, y-axis, and z-axis directionsshown in (a) of FIG. 3). Specifically, in the acceleration sensor 603,the long side direction of the housing 60 is defined as the x-axisdirection (in the state where the marker section 65 is placed above theLCD 61, the right direction along the long side direction when facingthe display screen of the LCD 61 is defined as an x-axis positivedirection), the short side direction of the housing 60 is defined as they-axis direction (in the state where the marker section 65 is placedabove the LCD 61, the up direction along the short side direction whenfacing the display screen of the LCD 61 is a y-axis positive direction),and the direction orthogonal to the front surface of the housing 60 isdefined as the z-axis direction (the perspective direction of thedisplay screen of the LCD 61 is defined as a z-axis positive direction),thereby detecting the magnitudes of the linear accelerations in therespective axis directions. Acceleration data representing the detectedaccelerations is output to the UI controller 605. The UI controller 605outputs, to the acceleration sensor 603, a control instruction for theacceleration sensor 603. In the exemplary embodiment, the accelerationsensor 603 is, for example, an electrostatic capacitance type MEMSacceleration sensor. In another embodiment, however, another type ofacceleration sensor may be used. Further, the acceleration sensor 603may be an acceleration sensor for detecting the magnitude ofacceleration in one axial direction or two axial directions.

The gyro sensor 604 is provided inside the housing 60. The gyro sensor604 detects the angular velocities about the three axes (the x, y, and zaxes described above). Angular velocity data representing the detectedangular velocities is output to the UI controller 605. The UI controller605 outputs, to the gyro sensor 604, a control instruction for the gyrosensor 604. Any number and any combination of gyro sensors may be usedas long as the angular velocities about three axes are detected. Thegyro sensor 604 may be constituted by a two-axis gyro sensor and aone-axis gyro sensor. Alternatively, the gyro sensor 604 may be a gyrosensor for detecting the angular velocity about one axis or two axes.

The vibrator 619 is, for example, a vibration motor or a solenoid. Thevibrator 619 is connected to the UI controller 605. The terminalapparatus 6 is vibrated by actuating the vibrator 619 in accordance witha control instruction outputted from the UI controller 605 to thevibrator 619. The vibration of the terminal apparatus 6 is transmittedto the user's hand holding the terminal apparatus 6. Thus, a so-calledvibration-feedback game is achieved.

The UI controller 605 outputs, to the codec LSI 606, the operation dataincluding the touch position data, the stick data, the operation buttondata, the orientation data, the acceleration data, and the angularvelocity data, which have been received from the respective components.If another device is connected to the terminal apparatus 6 through theextension connector 67, data representing operation to said anotherdevice may be included in the operation data.

The codec LSI 606 is a circuit for performing a compression process ondata to be transmitted to the game apparatus body 5, and a decompressionprocess on data transmitted from the game apparatus body 5. The LCD 61,the camera 66, the sound IC 608, the wireless module 610, the flashmemory 613, and the infrared communication module 612 are connected tothe codec LSI 606. The codec LSI 606 includes a CPU 617 and an internalmemory 618. Although the terminal apparatus 6 is configured not toperform game processing, the terminal apparatus 6 may execute a programfor managing the terminal apparatus 6 or a program for communication.For example, a program stored in the flash memory 613 is loaded into theinternal memory 618 and executed by the CPU 617 when the terminalapparatus 6 is powered on, thereby starting up the terminal apparatus 6.A part of the area of the internal memory 618 is used as a VRAM for theLCD 61.

The camera 66 takes an image in accordance with an instruction from thegame apparatus body 5, and outputs data of the taken image to the codecLSI 606. The codec LSI 606 outputs, to the camera 66, a controlinstruction for the camera 66, such as an instruction to take an image.The camera 66 is also capable of taking a moving picture. That is, thecamera 66 is capable of repeatedly performing image taking, andrepeatedly outputting image data to the codec LSI 606.

The sound IC 608 is connected to the loudspeakers 607 and the microphone609. The sound IC 608 is a circuit for controlling input of sound datafrom the microphone 609 to the codec LSI 606 and output of sound datafrom the codec LSI 606 to the loudspeakers 607. Specifically, when thesound IC 608 receives sound data from the codec LSI 606, the sound IC608 performs D/A conversion on the sound data, and outputs a resultantsound signal to the loudspeakers 607 to cause the loudspeakers 607 tooutput a sound. The microphone 609 detects sound (such as user's voice)propagated to the terminal apparatus 6, and outputs a sound signalrepresenting the sound to the sound IC 608. The sound IC 608 performsA/D conversion on the sound signal from the microphone 609, and outputsa predetermined form of sound data to the codec LSI 606.

The codec LSI 606 transmits the image data from the camera 66, the sounddata from the microphone 609, and the operation data from the UIcontroller 605 (terminal operation data), to the game apparatus body 5through the wireless module 610. In the exemplary embodiment, the codecLSI 606 subjects the image data and the sound data to a compressionprocess similar to that performed by the codec LSI 27. The compressedimage data and sound data, and the terminal operation data are output tothe wireless module 610 as transmission data. The antenna 611 isconnected to the wireless module 610, and the wireless module 610transmits the transmission data to the game apparatus body 5 through theantenna 611. The wireless module 610 has the same function as theterminal communication module 28 of the game apparatus body 5. That is,the wireless module 610 has a function of connecting to a wireless LANby a method based on, for example, the IEEE 802.11n standard. The datatransmitted from the wireless module 610 may be encrypted wherenecessary, or may not be encrypted.

As described above, the transmission data transmitted from the terminalapparatus 6 to the game apparatus body 5 includes the operation data(terminal operation data), the image data, and the sound data. Ifanother device is connected to the terminal apparatus 6 through theextension connector 67, data received from said another device may beincluded in the transmission data. The infrared communication module 612performs, with another device, infrared communication based on, forexample, the IRDA standard. The codec LSI 606 may include, in thetransmission data, data received by the infrared communication, andtransmit the transmission data to the game apparatus body 5, wherenecessary.

As described above, the compressed image data and sound data aretransmitted from the game apparatus body 5 to the terminal apparatus 6.These data are received by the codec LSI 606 through the antenna 611 andthe wireless module 610. The codec LSI 606 decompresses the receivedimage data and sound data. The decompressed image data is output to theLCD 61, and an image according to the image data is displayed on the LCD61. On the other hand, the decompressed sound data is output to thesound IC 608, and a sound based on the sound data is output from theloudspeakers 607.

When control data is included in the data received from the gameapparatus body 5, the codec LSI 606 and the UI controller 605 makecontrol instructions for the respective components, according to thecontrol data. As described above, the control data represents controlinstructions for the respective components (in the exemplary embodiment,the camera 66, the touch panel controller 601, the marker section 65,the sensors 602 to 604, the vibrator 619, and the infrared communicationmodule 612) included in the terminal apparatus 6. In the exemplaryembodiment, the control instructions represented by the control data areconsidered to be instructions to start and halt (stop) the operations ofthe above components. That is, some components which are not used for agame may be halted to reduce power consumption. In this case, data fromthe halted components are not included in the transmission datatransmitted from the terminal apparatus 6 to the game apparatus body 5.The marker section 65 is constituted by infrared LEDs, and therefore,the marker section 65 is controlled by simply turning on/off the supplyof power thereto.

As described above, the terminal apparatus 6 includes the operationmeans such as the touch panel 62, the analog sticks 63, and theoperation buttons 64. Alternatively, in another embodiment, the terminalapparatus 6 may include other operation means instead of or in additionto these operation means.

The terminal apparatus 6 includes the magnetic sensor 602, theacceleration sensor 603, and the gyro sensor 604 as sensors forcalculating the movement (including the position and the attitude, or achange in the position or the attitude) of the terminal apparatus 6.Alternatively, in another embodiment, the terminal apparatus 6 mayinclude one or two of these sensors. In still another embodiment, theterminal apparatus 6 may include other sensors instead of or in additionto these sensors.

The terminal apparatus 6 includes the camera 66 and the microphone 609.Alternatively, in another embodiment, the terminal apparatus 6 may notinclude the camera 66 and the microphone 609, or may include either ofthe cameral 66 and the microphone 609.

The terminal apparatus 6 includes the marker section 65 as a componentfor calculating the positional relation between the terminal apparatus 6and the controller 7 (such as the position and/or the attitude of theterminal apparatus 6 as viewed from the controller 7). Alternatively, inanother embodiment, the terminal apparatus 6 may not include the markersection 65. In still another embodiment, the terminal apparatus 6 mayinclude other means as a component for calculating the above positionalrelation. For example, the controller 7 may include a marker section,and the terminal apparatus 6 may include an image pickup element. Inthis case, the marker 8 may include an image pickup element instead ofan infrared LED.

Next, with reference to FIGS. 6 through 8, the configuration of theboard-type controller 9 is described. FIG. 6 is a perspective viewillustrating an example of the appearance of the board-type controller 9shown in FIG. 1. As shown in FIG. 6, the board-type controller 9includes a platform 9 a on which a user stands (on which the user placestheir feet), and at least four load sensors 94 a through 94 d fordetecting a load applied to the platform 9 a. Each of the load sensors94 a through 94 d is embedded in the platform 9 a (see FIG. 7), and thepositions where the load sensors 94 a through 94 d are provided areindicated by dotted lines in FIG. 6. In the following description, thefour load sensors 94 a through 94 d may be collectively referred to as aload sensor 94.

The platform 9 a is formed in the shape of substantially a rectangularparallelepiped, and is in the shape of substantially a rectangle asviewed from the top. For example, the short side of the rectangularshape of the platform 9 a is approximately 30 cm, and the long sidethereof is approximately 50 cm. The upper surface of the platform 9 a isflat, and has a pair of planes on which the user stands with the bottomsof their feet contacting thereto. Specifically, the upper surface of theplatform 9 a has a plane (a back-left region enclosed with a double linein FIG. 6) on which the user's left foot is placed, and a plane (afront-right region enclosed with a double line in FIG. 6) on which theuser's right foot is placed. The platform 9 a has, at four cornersthereof, side surfaces each partially projecting outward in acylindrical shape.

In the platform 9 a, the four load sensors 94 a through 94 d arearranged at predetermined intervals. In the exemplary embodiment, thefour load sensors 94 a through 94 d are arranged on the periphery of theplatform 9 a, more specifically, at the four corners of the platform 9a. The intervals of the load sensors 94 a through 94 d are appropriatelyset such that the load sensors 94 a through 94 d can accurately detectthe intention of a game operation which is expressed by a manner ofapplying a load to the platform 9 a by the user.

FIG. 7 shows an example of a cross-sectional view of the board-typecontroller 9, taken along line A-A in FIG. 6, and an example of anenlarged view of a corner part where a load sensor 94 is arranged. InFIG. 7, the platform 9 a includes a support plate 90 on which the userstands, and legs 92. The load sensors 94 a through 94 d are provided inpositions where the legs 92 are provided. In the exemplary embodiment,the four legs 92 are provided at the four corners, and therefore, thefour load sensors 94 a through 94 d are also provided at thecorresponding four corners. Each leg 92 is formed by plastic molding inthe shape of substantially a cylinder with a base. Each load sensor 94is located on a spherical part 92 a provided on the base of thecorresponding leg 92. The support plate 90 is supported by the legs 92via the load sensors 94.

The support plate 90 includes an upper plate 90 a forming an uppersurface and an upper side surface portion, a lower plate 90 b forming alower surface and a lower side surface portion, and an intermediateplate 90 c provided between the upper plate 90 a and the lower plate 90b. The upper plate 90 a and the lower plate 90 b are formed by, forexample, plastic molding, and are integrated using an adhesive or thelike. The intermediate plate 90 c is, for example, formed of a singlemetal plate by press forming. The intermediate plate 90 c is fixed ontothe four load sensors 94 a through 94 d. The upper plate 90 a has, on alower surface thereof, a grid-patterned rib (not shown), and issupported by the intermediate plate 90 c via the rib. Therefore, whenthe user stands on the platform 9 a, the load is transferred to the fourlegs 92 via the support plate 90 and the load sensors 94 a through 94 d.As indicated by arrows in FIG. 7, a reaction from a floor, which isgenerated by the input load, is transferred from the legs 92 through thespherical parts 92 a, the load sensors 94 a through 94 d and theintermediate plate 90 c to the upper plate 90 a.

Each load sensor 94 is, for example, a strain gauge (strain sensor) loadcell, which is a load converter for converting an input load to anelectrical signal. In the load sensor 94, a strain-generating body 95 isdeformed according to an input load, resulting in a strain. The strainis converted into a change of electrical resistance and then convertedinto a change of voltage by a strain sensor 96 attached to thestrain-generating body 95. Therefore, the load sensor 94 outputs, froman output terminal thereof, a voltage signal indicating the input load.

The load sensor 94 may be of other types, such as a tuning fork type, astring vibration type, an electrostatic capacitance type, apiezoelectric type, a magnetostrictive type, and a gyroscopic type.

Referring back to FIG. 6, the board-type controller 9 further includes apower button 9 c. When the power button 9 c is operated (e.g., when thepower button 9 c is pressed) in the state where the board-typecontroller 9 is not activated, power is supplied to each of circuitcomponents (see FIG. 8) of the board-type controller 9. There are,however, cases in which the board-type controller 9 is powered on inaccordance with an instruction from the game apparatus body 5 andthereby supply of power to the circuit components is started. Theboard-type controller 9 may be automatically powered off when a statewhere the user does not stand thereon continues for a predeterminedperiod of time (e.g., 30 sec) or more. Further, when the power button 9c is again operated in the state where the board-type controller 9 is inthe active state, the board-type controller 9 may be powered off to stopsupply of power to the circuit components.

FIG. 8 is a block diagram showing an example of an electricalconfiguration of the board-type controller 9. In FIG. 8, flows ofsignals and data are indicated by solid arrows, and supply of power isindicated by dotted arrows.

As shown in FIG. 8, the board-type controller 9 includes a microcomputer100 for controlling the operation thereof. The microcomputer 100includes a CPU, a ROM, a RAM, and the like, which are not shown. The CPUcontrols the operation of the board-type controller 9 in accordance witha program stored in the ROM.

The power button 9 c, an AD converter 102, a DC-DC converter 104, and awireless module 106 are connected to the microcomputer 100. An antenna106 a is connected to the wireless module 106. The four load sensors 94a through 94 d are connected to the AD converter 102 via amplifiers 108.

Further, the board-type controller 9 includes a battery 110 forsupplying power to the circuit components. In another embodiment, an ACadapter may be connected to the board-type controller 9 instead of thebattery 110 so that commercial power is supplied to the circuitcomponents. In this case, instead of the DC-DC converter 104, a powercircuit, which converts alternating current into direct current andlowers and rectifies a direct-current voltage, needs to be provided inthe board-type controller 9. In the exemplary embodiment, power issupplied directly from the battery 110 to the microcomputer 100 and thewireless module 106. In other words, power is constantly supplied fromthe battery 110 to the wireless module 106 and some components (such asthe CPU) in the microcomputer 100 to detect whether or not the powerbutton 9 c is turned on and whether or not a command that instructspower-on is transmitted from the game apparatus body 5. On the otherhand, power is supplied from the battery 110 through the DC-DC converter104 to the load sensors 94 a through 94 d, the AD converter 102, and theamplifiers 108. The DC-DC converter 104 converts a voltage value ofdirect current supplied from the battery 110 into a different voltagevalue, and supplies the resultant direct current to the load sensors 94a through 94 d, the AD converter 102, and the amplifiers 108.

Supply of power to the load sensors 94 a through 94 d, the A/D converter102 and the amplifiers 108 may be performed where necessary by themicrocomputer 100 that controls the DC-DC converter 104. Specifically,when the microcomputer 100 determines that it is necessary to operatethe load sensors 94 a through 94 d to detect a load, the microcomputer100 may control the DC-DC converter 104 to supply power to the loadsensors 94 a through 94 d, the A/D converter 102 and the amplifiers 108.

When power is supplied to the load sensors 94 a through 94 d, the loadsensors 94 a through 94 d each output a signal indicating a loadinputted thereto. These signals are amplified by the respectiveamplifiers 108, and converted from analog signals into digital data bythe A/D converter 102. The digital data is input to the microcomputer100. The detected values of the load sensors 94 a through 94 d are givenidentification information of the load sensors 94 a through 94 d, sothat the load sensors 94 a through 94 d can be identified from thecorresponding detected values. Thus, the microcomputer 100 can acquirethe data indicating the detected load values of the four load sensors 94a through 94 d at the same time.

On the other hand, when the microcomputer 100 determines that it is notnecessary to operate the load sensors 94 a through 94 d, i.e., when itis not the time for load detection, the microcomputer 100 controls theDC-DC converter 104 to stop supply of power to the load sensors 94 athrough 94 d, the A/D converter 102, and the amplifiers 108. Thus, theboard-type controller 9 can operate the load sensors 94 a through 94 dto detect a load or a distance only when it is required, resulting in areduction in power consumption for load detection.

Load detection is typically required when the game apparatus body 5(FIG. 1) needs to acquire load data. For example, when game apparatusbody 5 requires load information, the game apparatus body 5 transmits aninformation acquisition command to the board-type controller 9. When themicrocomputer 100 receives the information acquisition command from thegame apparatus body 5, the microcomputer 100 controls the DC-DCconverter 104 to supply power to the load sensors 94 a through 94 d andthe like, thereby detecting a load. On the other hand, when themicrocomputer 100 does not receive a load acquisition command from thegame apparatus body 5, the microcomputer 100 controls the DC-DCconverter 104 to stop supply of power to the load sensors 94 a through94 d and the like.

The microcomputer 100 may control the DC-DC converter 104 on the basisof a determination that the time of load detection arrives atpredetermined intervals. When such periodic load detection is performed,information regarding the constant time period may be supplied andstored from the game apparatus body 5 to the microcomputer 100 of theboard-type controller 9 when the game is started, or it may bepreinstalled in the microcomputer 100.

The data indicating the detected values from the load sensors 94 athrough 94 d are transmitted as board operation data (input data) forthe board-type controller 9 from the microcomputer 100 via the radiomodule 106 and an antenna 106 b to the game apparatus body 5. Forexample, when the microcomputer 100 has performed load detectionaccording to a command from the game apparatus body 5, the microcomputer100 transmits the detected value data of the load sensors 94 a through94 d to the game apparatus body 5 on receipt of the detected value datafrom the A/D converter 102. The microcomputer 100 may transmit thedetected value data to the game apparatus body 5 at predeterminedintervals. If the interval of the data transmission is longer than theinterval of the load detection, data containing load values which havebeen detected at a plurality of detection times up to the subsequenttime of transmission may be transmitted.

The wireless module 106 is set so as to perform communication accordingto the same wireless standard (the Bluetooth, wireless LAN, and thelike) as that for the controller communication module 19 of the gameapparatus body 5. Accordingly, the CPU 10 of the game apparatus body 5is allowed to transmit an information acquisition command to theboard-type controller 9 through the controller communication module 19and the like. Thus, the board-type controller 9 is allowed to receivethe command from the game apparatus body 5 through the wireless module106 and the antenna 106 a. Further, the board-type controller 9 isallowed to transmit the board operation data including the loaddetection values (or load calculation values) of the load sensors 94 athrough 94 d to the game apparatus body 5.

For example, in a game which is performed on the basis of a simple sumof four load values detected by the four load sensors 94 a through 94 d,the user is allowed to stand at a given position with respect to thefour load sensors 94 a through 94 d of the board-type controller 9. Thatis, the user is allowed to stand on the platform 9 a at a given positionand in a given direction to play a game. In some kinds of games,however, the direction of a load value detected by each of the four loadsensors 94 viewed from the user needs to be identified. That is, apositional relation between the four load sensors 94 of the board-typecontroller 9 and the user needs to be recognized. In this case, forexample, the positional relation between the four load sensors 94 andthe user may be defined in advance, and the user may be supposed tostand on the platform 9 a in a manner which allows the predeterminedpositional relation. Typically, a positional relation in which two ofthe load sensors 94 a through 94 d are present in front of, behind, tothe right of, and to the left of the user standing in the center of theplatform 9 a, i.e., a positional relation in which the user stands inthe center of the platform 9 a of the board-type controller 9, isdefined. In this case, the platform 9 a of the board-type controller 9is rectangular in shape as viewed from the top, and the power button 9 cis provided at one side (long side) of the rectangle. Therefore, it isruled in advance that the user, using the power button 9 c as a guide,stands on the platform 9 a such that the long side at which the powerbutton 9 c is provided is located in a predetermined direction (front,rear, left or right). In this case, each of the load values detected bythe load sensors 94 a through 94 d is a load value of a predetermineddirection (front right, front left, rear right, or rear left) as viewedfrom the user. Therefore, the board-type controller 9 and the gameapparatus body 5 can find out a direction to which each detected loadvalue corresponds as viewed from the user, on the basis of theidentification information of the load sensors 94 contained in thedetected load value data, and arrangement data indicating the positionsor the directions of the load sensors 94 with respect to the user thatis set (stored) in advance. As a result, it is possible to understandthe intention of a game operation performed by the user, such as anoperating direction, for example, a forward, a backward, or a leftward,a rightward direction, or a user's foot being lifted.

Next, with reference to the drawings, a description is given of anoverview of the game processing performed by the game apparatus body 5,before descriptions are given of specific processes performed by thegame apparatus body 5. It should be noted that FIG. 9 is a diagramshowing an example of the state of a user performing an operation usingthe terminal apparatus 6 and the board-type controller 9. FIG. 10A is adiagram showing an example of an image displayed on the LCD 61 of theterminal apparatus 6. FIG. 10B is a diagram showing an example of animage displayed on the monitor 2. FIG. 11 is a diagram showing anexample where the terminal apparatus 6 has been rotated (yawed) to theleft and right, and an example of an image displayed on the LCD 61. FIG.12 is a diagram illustrating examples of: the relationship between aterminal apparatus perspective direction projected onto a horizontalplane in real space and an operation indication direction projected ontoa horizontal plane in a virtual world; and a player object Po controlledso as to be directed in a direction based on the operation indicationdirection. FIG. 13 is a diagram illustrating examples of: the operationindication direction obtained by rotating (yawing) the terminalapparatus 6 to the left and right; and the player object Po controlledso as to be directed in a direction based on the operation indicationdirection. FIG. 14A is a diagram illustrating an example of a barrelleft-right operation range and a virtual camera left-right operationrange that are set in the left-right direction in the virtual world (orin real space). FIG. 14B is a diagram illustrating an example of abarrel up-down operation range and a virtual camera up-down operationrange that are set in the up-down direction in the virtual world (or inreal space). FIG. 15A is a diagram showing an example of a dirt image inwhich dirt clumps Bd are represented so as to be attached to the playerobject Po. FIG. 15B is a diagram showing an example of an image in whichareas in the dirt clumps Bd represented so as to be attached to theplayer object Po are removed by a touch operation.

As shown in FIG. 9, the user performs an operation using the terminalapparatus 6 and the board-type controller 9. The user performs theoperation of changing the attitude and the direction of the terminalapparatus 6, the operation of touching the touch panel 62 of theterminal apparatus 6, and the operation of changing a load to be appliedto the board-type controller 9. Specifically, the user places one footon the board-type controller 9 while holding the terminal apparatus 6.Then, the user plays by taking action on the board-type controller 9while viewing an image displayed on the monitor 2 or an image displayedon the LCD 61 of the terminal apparatus 6 (e.g., performing theoperation of taking action so as to step with the one foot placed on theboard-type controller 9, thereby increasing and decreasing a weight tobe put on the one foot placed on the board-type controller 9), and alsoperforming the operation of moving the terminal apparatus 6 andperforming the operation of touching the touch panel 62 of the terminalapparatus 6. Then, on the LCD 61 and the monitor 2 of the terminalapparatus 6, game images are represented such that a player object Potakes action in a virtual world (e.g., the action of changing itsdirection, and the action of discharging a discharge object) inaccordance with the direction and the attitude of the terminal apparatus6 held by the user and the action taken by the user on the board-typecontroller 9, and the attitude of a virtual camera set in the virtualworld is changed in accordance with the direction of the player objectPo. Further, it is possible to perform an operation on dirt clumps Bdrepresented so as to be attached to the player object Po, by performinga touch operation on the touch panel 62 of the terminal apparatus 6.

As shown in FIG. 10A, on the LCD 61 of the terminal apparatus 6, thestate of a player object Po shooting a water cannon in a virtual worldis displayed from the first-person point of view of the player objectPo. In the example shown in FIG. 10A, the virtual world viewed from thefirst-person point of view is displayed that includes an end portion ofthe water cannon (an end portion of a barrel) operated by the playerobject Po, and the state of the water cannon discharging water W, whichis an example of a discharge object, is displayed. Further, a pluralityof enemy objects Eo are also placed in the virtual world, and the stateof one of the enemy objects Eo throwing an enemy bomb object B at theplayer object Po is displayed. The virtual world viewed from thefirst-person point of view of the player object Po is thus displayed onthe LCD 61, whereby the user, viewing the display on the LCD 61 whileholding the terminal apparatus 6, can play a game from the same point ofview as that of the player object Po. This makes it possible to providea sense of presence in the virtual world.

In addition, as shown in FIG. 10B, also on the monitor 2, the samevirtual world as the virtual world displayed on the LCD 61 is displayed.In the example shown in FIG. 10B, the state of the virtual world viewedfrom a position behind, above, and far from the player object Pooperating the water cannon is displayed together with the player objectPo. The state of the virtual world viewed from a position behind, above,and far from the player object Po is thus displayed on the monitor 2,whereby the user can easily understand the circumstance of the playerobject Po and the positional relationships among the player object Poand the enemy objects Eo, and another person viewing the state of theuser playing the game can also enjoy viewing the attacking action of theplayer object Po.

It should be noted that in the example shown in FIG. 10B, on the monitor2, the state of the virtual world is displayed that is viewed from aposition behind, above, and far from the player object Po.Alternatively, the virtual world viewed from another point of view maybe displayed on the monitor 2. The same virtual world may be displayednot only on the terminal apparatus 6 but also on the monitor 2, andimages of the virtual world that are different from each other in thepoint of view may be displayed, whereby, in accordance with the state ofthe operation or preference, the user can appropriately use either oneof the images displayed on the two display apparatuses when performingan operation. For example, if, in contrast to the image viewed from thefirst-person point of view of the player object Po and displayed on theterminal apparatus 6, a virtual camera (second virtual camera) fordisplaying the virtual world on the monitor 2 is set at a position awayfrom the player object Po so that a range wider than the range of thevirtual world displayed on the terminal apparatus 6 is displayed on themonitor 2, the position of the virtual camera may not need to be behind,above, and far from the player object Po. Specifically, the virtualcamera for displaying the virtual world on the monitor 2 may be set at aposition of viewing the player object Po from a bird's-eye view or aposition of looking down upon it.

It should be noted that in the example shown in FIG. 10B, on the monitor2, the state of the virtual world is displayed that is viewed from aposition behind, above, and far from the player object Po.Alternatively, the virtual world viewed from another point of view maybe displayed on the monitor 2. The same virtual world may be displayednot only on the terminal apparatus 6 but also on the monitor 2, andimages of the virtual world that are different from each other in thepoint of view may be displayed, whereby, in accordance with the state ofthe operation or preference, the user can appropriately use either oneof the images displayed on the two display apparatuses when performingan operation. For example, if, in contrast to the image viewed from thefirst-person point of view of the player object Po and displayed on theterminal apparatus 6, a virtual camera (second virtual camera) fordisplaying the virtual world on the monitor 2 is set at a position awayfrom the player object Po so that a range wider than the range of thevirtual world displayed on the terminal apparatus 6 is displayed on themonitor 2, the position of the virtual camera may not need to be behind,above, and far from the player object Po. Specifically, the virtualcamera for displaying the virtual world on the monitor 2 may be set at aposition of viewing the player object Po from a bird's-eye view or aposition of looking down upon it.

For example, as described above, the board-type controller 9 outputsdetected load values based on the action taken by the user on theboard-type controller 9. Then, the use of the detected load values makesit possible to calculate the total load applied to the board-typecontroller 9. The use of the total load makes it possible to estimatewhether the user is putting weight on the board-type controller 9, or isdecreasing the weight put on the board-type controller 9. Further, theuse of the total load also makes it possible to calculate the magnitudeof the load applied to the board-type controller 9 by the user, and theamount of change in the load applied to the board-type controller 9. Theaction of the player object Po discharging the discharge object is setin accordance with the action of the user thus estimated on theboard-type controller 9.

In addition, in accordance with the attitude (direction) of the terminalapparatus 6 held by the user, the direction in which the player objectPo views the virtual world (i.e., the forward direction of the playerobject Po in the virtual world; the direction of the line of sight of avirtual camera placed at the first-person point of view of the playerobject Po) changes, and also the direction in which the player object Podischarges the discharge object (e.g., the water W) (the direction ofthe barrel of the water cannon) changes. For example, in accordance withthe user directing the back surface of the terminal apparatus 6 upward,downward, leftward, and rightward, that is, directing the z-axispositive direction, which is the perspective direction of the LCD 61 (aterminal apparatus perspective direction), upward, downward, leftward,and rightward, the direction of the player object Po in the virtualworld changes upward and leftward and to the left and right, and alsothe direction in which the water cannon discharges the discharge objectchanges upward and leftward and to the left and right in the virtualworld. Further, in accordance with the user directing the terminalapparatus perspective direction of the terminal apparatus 6 upward,downward, leftward, and rightward, also the direction of the line ofsight of the virtual camera changes upward and leftward and to the leftand right. Consequently, also a game image displayed on the LCD 61 andviewed from the first-person point of view of the player object Pochanges in accordance with the change in the direction of the line ofsight. For example, as shown in FIG. 11, when the user has changed thedirection of the terminal apparatus 6 such that the terminal apparatusperspective direction is directed rightward, the direction of the playerobject Po and the direction of the barrel of the water cannon in thevirtual world change to the right, and also the direction of the line ofsight of the virtual camera in the virtual world changes to the right bythe same angle. As is clear by comparing FIGS. 11 and 10A, this resultsin causing the virtual world to be displayed on the LCD 61 so as toscroll to the left, and causing the barrel of the water cannon to bedisplayed at a fixed position on the LCD 61 so as to have the sameattitude. Then, if the water W is continuing to be discharged from thebarrel of the water cannon, display is performed on the LCD 61 such thatthe water W is discharged in a meandering manner in the virtual world inaccordance with the change in the direction of the barrel.

FIGS. 12 and 13 each show the attitude of the terminal apparatus 6 thatis obtained by looking down upon real space, and the attitudes of theplayer object Po and the virtual camera that are obtained by lookingdown upon the virtual world. As shown in FIG. 12, a virtual camera(first virtual camera) for generating the virtual world to be displayedon the LCD 61 is placed at the first-person point of view of the playerobject Po that operates the water cannon in the virtual world. Then, anoperation indication direction is calculated by reflecting on thevirtual world the direction of the terminal apparatus perspectivedirection of the terminal apparatus 6 (the z-axis positive direction) inreal space, and the direction of the player object Po and the directionof the barrel are set so as to coincide with the operation indicationdirection. Further, the attitude of the virtual camera is controlledsuch that the direction that coincides with the operation indicationdirection (i.e., the direction of the barrel) is the direction of theline of sight of the virtual camera (the Z-direction shown in thefigures). The operation indication direction obtained by reflecting theterminal apparatus perspective direction on the virtual world is thusset so as to coincide with the direction of the line of sight of thevirtual camera, whereby the direction in which the terminal apparatus 6is directed upward, downward, leftward, and rightward in real spacecoincides with the direction in which the virtual camera is directedupward, downward, leftward, and rightward in the virtual world. Thismakes it possible to display on the LCD 61 an image as if peeping at thevirtual world using the LCD 61 as a peep window.

The case is considered where the user has changed the direction of theterminal apparatus 6 such that the terminal apparatus perspectivedirection is directed rightward (the direction A shown in FIG. 12). Forexample, as shown in FIG. 13, the case is considered where the directionof the terminal apparatus 6 has changed such that the terminal apparatusperspective direction is directed in the direction A and by an angle B.In this case, the operation indication direction changes in thedirection A and by the angle B also in the virtual world, in a similarmanner to the change in the terminal apparatus perspective direction inreal space. Then, also the direction of the player object Po and thedirection of the barrel of the water cannon operated by the playerobject Po change about a predetermined position in the virtual world(e.g., the position of the player object Po, i.e., the position of thefirst-person point of view where the virtual camera is placed) in thedirection A, which is the same as that of the change in the operationindication direction, and by the angle B. Further, also the direction ofthe line of sight of the virtual camera changes about a predeterminedposition in the virtual world (e.g., the position of the point of viewof the virtual camera) in the direction A, which is the same as that ofthe change in the operation indication direction, and by the angle B.

Here, the range where the user is allowed to change the direction of theplayer object Po and the direction of the barrel of the water cannon maybe limited in a predetermined range in advance. For example, as shown inFIG. 14A, a barrel left-right operation range, where the barrel canchange its direction to the left and right in the virtual world, is setto a predetermined angular range about a virtual world referencedirection (e.g., a range of 90° in total, which includes 45° to both theleft and right of the virtual world reference direction, or a range of180° in total, which includes 90° to both the left and right of thevirtual world reference direction). It should be noted that the virtualworld reference direction is a direction indicating the forwarddirection of the virtual world that corresponds to the forward directionof the user in real space (a real space reference direction), and isset, as an example, in accordance with an operation of the user.Further, as shown in FIG. 14B, a barrel up-down operation range, wherethe barrel can change its direction upward and downward in the virtualworld, is set to a predetermined angular range with respect to thehorizontal direction in the virtual world (the horizontal direction inreal space) (e.g., a range of 55° in total, which includes 45° in theelevation direction from the horizontal direction in the virtual worldand 10° in the depression direction from the horizontal direction in thevirtual world). Then, if the operation indication direction is setoutside the barrel left-right operation range and/or outside the barrelup-down operation range, the direction of the barrel is set in thebarrel left-right operation range and/or in the barrel up-down operationrange so as to be closest to the operation indication direction.

On the other hand, the range where the user is allowed to change thedirection of the line of sight of the virtual camera may not need to belimited. For example, as shown in FIG. 14A, a virtual camera left-rightoperation range, where the direction of the line of sight of the virtualcamera can be changed to the left and right, can be set in alldirections. Further, as shown in FIG. 14B, also a virtual camera up-downoperation range, where the direction of the line of sight of the virtualcamera can be changed upward and downward, can be set in all directions.Accordingly, if the operation indication direction is set outside thebarrel left-right operation range and/or outside the barrel up-downoperation range, the direction of the barrel is set in the barrelleft-right operation range and/or in the barrel up-down operation range,while the direction of the line of sight of the virtual camera is set soas to be the same as the operation indication direction. That is, whenthe user has directed the terminal apparatus 6 in the direction in whichthe operation indication direction is calculated so as to be outside thebarrel left-right operation range and/or outside the barrel up-downoperation range, the virtual world is displayed on the LCD 61 such thatthe direction of the line of sight of the virtual camera is differentfrom the forward direction of the player object Po and the direction ofthe barrel.

For example, acceleration data or angular velocity data based on themotion and a change in the attitude of the terminal apparatus 6 isoutput from the terminal apparatus 6. Then, the direction of thegravitational acceleration applied to the terminal apparatus 6 can becalculated using the acceleration indicated by the acceleration data.This makes it possible to estimate the attitude of the terminalapparatus 6 with respect to the vertical direction in real space.Further, the use of the angular velocity and/or the dynamic accelerationapplied to the terminal apparatus 6 using the angular velocity indicatedby the angular velocity data and/or the acceleration indicated by theacceleration data, makes it possible to estimate a change in theattitude of the terminal apparatus from its initial attitude in realspace (i.e., a change in direction) using the angular velocity and/orthe dynamic acceleration. In accordance with the thus estimated changein the attitude of the terminal apparatus 6 (a change in direction), theaction of the player object Po (the forward direction of the playerobject Po and the direction of the barrel) and the attitude (thedirection of the line of sight) of the virtual camera are set.

As described above, the user can change the action of the player objectPo and the attitude of the virtual camera on the basis of the directionand the attitude of the terminal apparatus 6 held by the user. Forexample, in accordance with the direction and the attitude of theterminal apparatus 6 held by the user, the direction of the playerobject Po and the direction of the barrel change, and also the directionin which the discharge object is to be discharged from the barrel (adischarge direction) changes. As an example, as a result of the userdirecting the terminal apparatus 6 upward, downward, leftward, andrightward (i.e., pitching and yawing the terminal apparatus 6), thedirection of the player object Po and the direction of the barrel changein conjunction with the change in the direction of the terminalapparatus 6, and also the discharge direction changes. Specifically,when the user has changed the direction of the terminal apparatus 6 soas to direct the back surface of the terminal apparatus 6 upward (i.e.,pitch the terminal apparatus 6 in the elevation direction), thedirection of the player object Po and the direction of the barrel changeupward in the virtual world within the barrel up-down operation range.Further, when the user has changed the direction of the terminalapparatus 6 so as to direct the back surface of the terminal apparatus 6leftward (i.e., yaw the terminal apparatus 6 to the left), the directionof the player object Po and the direction of the barrel change to theleft in the virtual world within the barrel left-right operation range.By thus bringing the attitude and the direction of the terminalapparatus 6 in conjunction with the direction of the barrel, the usercan perform an operation having verisimilitude as if the user themselvesis moving the water cannon (barrel) using the terminal apparatus 6.Further, as described above, the virtual camera is set at thefirst-person point of view of the player object Po that operates thewater cannon, and the direction of the line of sight of the virtualcamera changes in accordance with the direction and the attitude of theterminal apparatus 6 held by the user. By thus bringing the attitude andthe direction of the terminal apparatus 6 in conjunction with theattitude and the direction of the virtual camera, the user can enjoy afeeling as if the user themselves is the player object Po that operatesthe water cannon, and can also enjoy a feeling as if peeping at thevirtual world through the LCD 61 of the terminal apparatus 6. Further,in the exemplary game described above, in accordance with the userapplying a load to the board-type controller 9, the action ofdischarging the discharge object from the water cannon is taken. Then,the details of the discharge object to be discharged (e.g., the presenceor absence of the discharge of the discharge object, the amount ofdischarge and the discharge velocity in and at which the dischargeobject is to be discharged, and the type of the discharge object) aredetermined in accordance with the load applied to the board-typecontroller 9, thereby enabling an analog operation. That is, the user isprovided, by an image displayed on the LCD 61, with a feeling as ifbeing in the virtual world, and is additionally provided, by an analogoperation using the board-type controller 9, with an operation feelingas if the user themselves is operating a water cannon in real space.This enhances the feeling as if being in the virtual world.

It should be noted that the player object Po may be caused to move inthe virtual world on the basis of the attitude and the motion of theterminal apparatus 6. For example, a moving angle and a moving distanceare calculated on the basis of changes in the attitude and the motion ofthe terminal apparatus 6. Then, the player object Po that operates thewater cannon is caused to move in the virtual world in accordance withthe moving angle and the moving distance, and the first virtual camerais also caused to move in accordance with the movement of the playerobject Po.

As shown in FIG. 15A, the case is considered where an enemy bomb objectB thrown by an enemy object Eo has hit the player object Po. Torepresent the state of the player object Po being soiled by the hittingof the enemy bomb object B, a dirt clump Bd based on the enemy bombobject B is represented so as to be attached on the surface of the LCD61. The attachment of the dirt clump Bd, as shown in FIG. 15A, hindersthe field of view toward the virtual world using the LCD 61. Thiscreates a disadvantage for the user to play the game.

It is possible to remove the dirt clump Bd attached to the surface ofthe LCD 61 by performing a touch operation on the touch panel 62 of theterminal apparatus 6. For example, as shown in FIG. 15B, when the userhas performed the touch operation on the touch panel 62 that covers thesurface of the LCD 61, areas in dirt clumps Bd attached to the surfaceof the LCD 61 are removed, the areas corresponding to the touchedposition. As an example, when the areas in the dirt clumps Bd areremoved, a predetermined range whose center is the touch position atwhich the user has touched the touch panel 62 is removed from thesurface of the LCD 61. Accordingly, when the user has performed thetouch operation so as to drag the touch panel 62 (e.g., performed thetouch operation so as to trace the dashed line shown in FIG. 15B), theareas in the dirt clumps Bd corresponding to the line on which the draghas been performed, or corresponding to an area having a predeterminedline width with respect to the line, are removed from the surface of theLCD 61. It should be noted that the action of removing the dirt clumpsBd attached to the surface of the LCD 61 can be considered as the actionof the player object Po removing the dirt clumps Bd attached to theplayer object Po itself. Thus, in this case, the user causes the playerobject Po to take action by operating the touch panel 62 of the terminalapparatus 6.

Next, the game processing performed by the game system 1 is described indetail. First, with reference to FIG. 15, main data used in the gameprocessing is described. FIG. 15 is a diagram showing an example of maindata and programs that are stored in a main memory of the game apparatusbody 5.

As shown in FIG. 16, in a data storage area of the main memory, thefollowing are stored: board operation data Da; terminal operation dataDb; load value data Dc; terminal apparatus direction/attitude data Dd;operation direction data De; barrel direction data Df; discharge objectdata Dg; enemy bomb object data Dh; virtual camera data Di; image dataDh; and the like. It should be noted that the main memory appropriatelystores, as well as the data shown in FIG. 16, data used for the gameprocessing, such as image data of various objects displayed on themonitor 2 and the LCD 61, and sound data used for the game. Further, ina program storage area of the main memory, various programs Pa includedin the game program are stored.

As the board operation data Da, a series of operation information (boardoperation data) transmitted as transmission data from the board-typecontroller 9 is stored, and updated to the latest board operation data.For example, the board operation data Da includes load data Da1 and thelike. The load data Da1 is data indicating load values detected by theload sensors 94 a through 94 d of the board-type controller 9.

As the terminal operation data Db, a series of operation information(terminal operation data) transmitted as transmission data from theterminal apparatus 6 is stored, and updated to the latest terminaloperation data. For example, the terminal operation data Db includesacceleration data Db1, angular velocity data Db2, touch position dataDb3, and the like. The acceleration data Db1 is data indicating anacceleration (an acceleration vector) detected by the accelerationsensor 603. For example, the acceleration data Db1 represents athree-dimensional acceleration vector whose components are accelerationsin the three axial (x-axis, y-axis, and z-axis) directions shown in FIG.3. In another embodiment, the acceleration data Db1 may representaccelerations in given one or more directions. The angular velocity dataDb2 is data representing an angular velocity detected by the gyro sensor604. For example, the angular velocity data Db2 represents angularvelocities about the three axes (x-axis, y-axis, and z-axis) shown inFIG. 3. In another example, the angular velocity data Db2 may representangular velocities about given one or more axes. The touch position dataDb3 is data representing the coordinates of the position at which aninput has been provided on the input surface of the touch panel 62.

It should be noted that the game apparatus body 5 sequentially receivesthe data (e.g., the data indicating the detected load values, theacceleration, and the angular velocity) included in the operationinformation transmitted from the controller 7, the board-type controller9, and the terminal apparatus 6 at predetermined intervals (e.g., atintervals of 1/200 seconds). For example, the received data issequentially stored in the main memory by the I/O processor 31. In aprocessing flow described later, the CPU 10 reads the latest boardoperation data and the latest terminal operation data from the mainmemory every frame period (e.g., 1/60 seconds), to thereby update theboard operation data Da and the terminal operation data Db.

In addition, the operation information transmitted from the controller7, the board-type controller 9, and the terminal apparatus 6 at thepredetermined intervals may be temporarily stored in the buffer (notshown) included in the controller communication module 19 or theterminal communication module 28. In this case, the data stored in thebuffer is read every frame period, and the board operation data Da(e.g., the load data Da1) or the terminal operation data Db (e.g., theacceleration data Db1, the angular velocity data Db2, and the touchposition data Db3) in the main memory is updated for use. At this time,the cycle of receiving the operation information is different from theprocessing cycle, and therefore, a plurality of pieces of informationreceived at a plurality of times are stored in the buffer. Theprocessing may be performed using only the latest operation informationamong the plurality of pieces of operation information received at theplurality of times. Alternatively, the processing may be performed usinga representative value (e.g., an average value) of the pieces ofoperation information received at the plurality of times. Yetalternatively, the processing may be performed multiple times so as tocorrespond to the number of the pieces of operation information receivedat the plurality of times.

The load value data Dc is an aggregate of data indicating the loadvalues detected by the board-type controller 9. For example, the loadvalue data Dc is an aggregate of data indicating the sum of the loadvalues (the total load value) detected by the load sensors 94 a through94 d. Specifically, the load value data Dc is an array of dataindicating the total load values within a predetermined period that arechronologically calculated, and the data indicating the total loadvalues is chronologically stored in the elements of the array.

The terminal apparatus direction/attitude data Dd includes real spacereference direction data Dd1, current direction data Dd2, and the like.The real space reference direction data Dd1 is data indicating areference direction (the attitude; the real space reference direction)of the terminal apparatus 6 in real space. The current direction dataDd2 is data indicating the current direction and attitude of theterminal apparatus 6 in real space. In the exemplary embodiment, thereal space reference direction data Dd1 and the current direction dataDd2 are subjected to various corrections when set. For example, the realspace reference direction data Dd1 and the current direction data Dd2are calculated on the basis of the acceleration data Db1 and the angularvelocity data Db2 that are included in the terminal operation data Db.It should be noted that the method of calculating the real spacereference direction and the current direction will be described later.

The operation direction data De includes virtual world referencedirection data De1, operation indication direction data De2, and thelike. The virtual world reference direction data De1 is data indicatingthe virtual world reference direction set in the virtual world. Theoperation indication direction data De2 is data indicating the operationindication direction currently indicated in the virtual world by theuser. It should be noted that the method of calculating the virtualworld reference direction and the operation indication direction will bedescribed later.

The barrel direction data Df includes barrel left-right direction dataDf1, barrel up-down direction data Df2, and the like. The barrelleft-right direction data Df1 is data indicating the left-rightdirection of the barrel of the water cannon in the virtual world. Thebarrel up-down direction data Df2 is data indicating the up-downdirection of the barrel of the water cannon in the virtual world.

The discharge object data Dg includes object type data Dg1, amount ofdischarge data Dg2, discharge vector data Dg3, position data Dg4, andthe like, for each discharge object present in the virtual world. Theobject type data Dg1 is data indicating the type (e.g., the water W orthe large ball) of the discharge object to be discharged from thebarrel. The amount of discharge data Dg2 is data indicating the amountof discharge per unit time in which the discharge object is to bedischarged from the barrel. The discharge vector data Dg3 is dataindicating the moving velocity and the moving direction of the dischargeobject discharged per unit time in the virtual world. The position dataDg4 is data indicating the position of the discharge object dischargedper unit time in the virtual world.

The enemy bomb object data Dh is data concerning an enemy bomb object Bthat is thrown at the player object Po by an enemy object Eo placed inthe virtual world, and is data indicating the type, the size, theposition, the moving velocity, the moving direction, and the like of theenemy bomb object B.

The virtual camera data Di is data concerning virtual cameras set in thevirtual world. For example, the virtual camera data Di includes dataconcerning a first virtual camera for generating a game image to bedisplayed on the LCD 61 of the terminal apparatus 6, and data concerninga second virtual camera for generating a game image to be displayed onthe monitor 2.

The image data Dj includes player object data Dj1, discharge objectimage data Dj2, dirt image data Dj3, background image data Di4, and thelike. The player object data Dj1 is data for placing in the virtualworld the player object Po that operates the water cannon, to generate agame image. The discharge object image data Dj2 is data for placing thedischarge object in the virtual world to generate a game image. The dirtimage data Dj3 is data indicating an image in which a dirt clump Bdbased on an enemy bomb object B is represented so as to be attached tothe surface of the LCD 61. The dirt image data Dj3 is data indicating animage to be displayed on the LCD 61 in combination with an image of thevirtual world to be displayed on the LCD 61. The background image dataDj4 is data for placing background in the virtual world to generate agame image.

Next, with reference to FIGS. 17 through 23, the game processingperformed by the game apparatus body 5 is described in detail. It shouldbe noted that FIG. 17 is a flow chart showing an example of the gameprocessing performed by the game apparatus body 5. FIG. 18 is asubroutine flow chart showing an example of a game control process instep 44 in FIG. 17. FIG. 19 is a subroutine flow chart showing anexample of a player object setting process in step 83 in FIG. 18. FIG.20 is a subroutine flow chart showing an example of an operationindication direction calculation process in step 121 in FIG. 19. FIG. 21is a subroutine flow chart showing an example of a discharge objectsetting process in step 130 of FIG. 19. FIG. 22 is a subroutine flowchart showing an example of an attack reception operation in step 131 ofFIG. 19. FIG. 23 is a diagram illustrating an example of movementvectors Vw1 through Vw15 respectively set for discharge objects W1through W15 that move in the virtual world. Here, in the flow chartsshown in FIGS. 17 through 22, descriptions are given mainly of, amongthe processes of the game processing, a process where the player objectPo is displayed so as to move in accordance with the operation performedby the user using the terminal apparatus 6 and the board-type controller9, while detailed descriptions of the other processes not directlyrelated to the exemplary embodiment are omitted. Further, in FIGS. 17through 22, each step performed by the CPU 10 is abbreviated as “S”.

When the game apparatus body 5 has been powered on, the CPU 10 of thegame apparatus body 5 executes a boot program stored in the ROM/RTC 13to initialize each unit such as the main memory. Then, the game programstored in the optical disk 4 is loaded to the main memory, and the CPU10 starts to execute the program. The flow charts shown in FIGS. 16through 19 show processes to be performed after the above processes arecompleted.

Referring to FIG. 16, the CPU 10 performs an initialization process(step 40), and proceeds to the subsequent step. For example, in theinitialization process in step 40, the CPU 10 constructs the virtualworld, places the player object Po and the virtual cameras (the firstvirtual camera and the second virtual camera) in the virtual world atpredetermined positions, places objects at initial positions, and setsthe initial values of various parameters used for the game processing.

Next, the CPU 10 sets a reference direction on the basis of datatransmitted from the terminal apparatus 6 (step 41), and proceeds to thesubsequent step. A description is given below of an example where theCPU 10 sets the reference direction.

The terminal apparatus 6 repeatedly transmits data as described above tothe game apparatus body 5. In the game apparatus body 5, the terminalcommunication module 28 sequentially receives the data described above,and the I/O processor 31 sequentially stores terminal operation data,camera image data, and microphone sound data in the main memory. In step41 described above, the CPU 10 reads the most recent terminal operationdata from the main memory, to thereby update the acceleration data Db1,the angular velocity data Db2, and the touch position data Db3.

Next, the CPU 10 calculates the direction and the attitude of theterminal apparatus 6 in real space. For example, the CPU 10 calculates,as the reference direction (initial attitude) in real space, the currentdirection and attitude of the terminal apparatus 6 on the basis of theacceleration indicated by the acceleration data Db1 and the angularvelocity indicated by the angular velocity data Db2, to thereby updatethe real space reference direction data Dd1 using data indicating thecalculated reference direction of the terminal apparatus 6. For example,the CPU 10 can calculate the amount of rotation (the amount of change inthe direction) of the terminal apparatus 6 in real space per unit time,using the angular velocity indicated by the angular velocity data Db2.Further, in the state where the terminal apparatus 6 is substantiallystationary (in a static state) in real space, the acceleration appliedto the terminal apparatus 6 is the gravitational acceleration. Thismakes it possible to calculate the direction of gravity applied to theterminal apparatus 6 (i.e., the attitude of the terminal apparatus 6with respect to the vertical direction in real space), using theacceleration indicated by the acceleration data Db1. This enables theCPU 10 to calculate the initial attitude of the terminal apparatus 6 onthe basis of the acceleration indicated by the acceleration data Db1 andthe angular velocity indicated by the angular velocity data Db2. Itshould be noted that in the following descriptions, when step 41described above is performed, the real space reference direction is seton the basis of the direction in which the back surface of the terminalapparatus 6 is directed in real space (the z-axis positive directionshown in FIG. 3, i.e., the terminal apparatus perspective direction).

It should be noted that the initial attitude of the terminal apparatus 6may be calculated on the basis of the acceleration indicated by theacceleration data Db1, or may be calculated on the basis of thedirection of magnetism detected by the magnetic sensor 602.Alternatively, as a result of the user performing a predeterminedoperation in the state where the terminal apparatus 6 is in a specificattitude, the specific attitude when the predetermined operation hasbeen performed may be used as the initial attitude. It should be notedthat the initial attitude needs to be calculated if the attitude of theterminal apparatus 6 is calculated as an absolute attitude with respectto a predetermined direction in real space. Timing may be set such thatthe setting of the initial attitude, that is, step 41 described above,is performed at the start of the game, or is performed in accordancewith a predetermined operation performed by the user using the terminalapparatus 6 (e.g., the operation of pressing a predetermined operationbutton 64).

In addition, in step 41 described above, the real space referencedirection is transformed into that of a model coordinate system in thevirtual world, whereby the virtual world reference direction data De1 isupdated using the direction after the transformation as the referencedirection in the virtual world.

It should be noted that in the setting process of the referencedirection in step 41 described above, the reference direction is setafter the attitude and the direction of the terminal apparatus 6 aresubjected to various corrections, whereby the real space referencedirection data Dd1 is updated using the reference direction after thecorrections. Further, the real space reference direction after thecorrections is transformed into that of the model coordinate system inthe virtual world, whereby the virtual world reference direction dataDe1 is updated using the direction after the transformation as thereference direction in the virtual world. It should be noted that adescription will be given later of the method of correcting the attitudeand the direction of the terminal apparatus 6.

Subsequent to step 41 described above, the process in step 42 isperformed. Thereafter, the processing loop of a series of processes 42through 51 is performed every predetermined period (one frame period)and repeated.

In step 42, the CPU 10 acquires board operation data transmitted fromthe board-type controller 9, and proceeds to the subsequent step. Here,the board-type controller 9 repeatedly transmits the board operationdata to the game apparatus body 5. Accordingly, in the game apparatusbody 5, the controller communication module 19 sequentially receives theboard operation data, and the I/O processor 31 sequentially stores thereceived board operation data in the main memory. The interval oftransmission of the board operation data from the board-type controller9 may be shorter than the game processing period (one frame period), andit is 1/200 seconds, for example. In step 42, the CPU 10 reads thelatest board operation data from the main memory, to thereby update theboard operation data Da. The board operation data includes dataindicating identification information of the load sensors 94 a through94 d, and data indicating the load values detected by the load sensors94 a through 94 d. The load data Da1 is updated using the dataidentified by the identification information.

Next, the CPU 10 acquires various data transmitted from the terminalapparatus 6 (step 43), and proceeds to the subsequent step. The terminalapparatus 6 repeatedly transmits the data to the game apparatus body 5.Accordingly, in the game apparatus body 5, the terminal communicationmodule 28 sequentially receives the data, and the codec LSI 27sequentially performs a decompression process on the camera image dataand the microphone sound data. Then, the I/O processor 31 sequentiallystores the terminal operation data, the camera image data, and themicrophone sound data in the main memory. In step 43 described above,the CPU 10 reads the latest terminal operation data from the mainmemory, to thereby update the acceleration data Db1, the angularvelocity data Db2, and the touch position data Db3.

Next, the CPU 10 performs a game control process (step 44), and proceedsto the subsequent step. The game control process is the process of, forexample, causing the player object Po and the virtual camera in thevirtual world to move in accordance with a game operation performed bythe user, to thereby advance the game. In this exemplary game, the useris allowed to play various games using the terminal apparatus 6 and theboard-type controller 9. With reference to FIG. 18, a description isgiven below of the game control process in step 44 described above.

In FIG. 18, the CPU 10 calculates a load value (step 81), and proceedsto the subsequent step. For example, the CPU 10 calculates a total loadvalue by summing up the detected load values indicated by the load dataDa1, to thereby update the latest data in the chronological data arrayof the load value data Dc, using the data indicating the calculatedtotal load value. Specifically, the load data Da1 indicates the latestload values detected by the load sensors 94 a through 94 d, andtherefore, the total load value is calculated by summing up the detectedload values. The thus calculated total load value changes in accordancewith the action taken by the user and the shifting of their weight(attitude) on the board-type controller 9. As an example, when the userhas taken action so as to apply a load to the board-type controller 9,the total load value increases in accordance with the applied load.

Next, the CPU 10 calculates a change in the direction and the attitudeof the terminal apparatus 6 (step 82), and proceeds to the subsequentstep. For example, the CPU 10 calculates the x-axis, y-axis, and z-axisdirections of the terminal apparatus 6 in real space on the basis of theacceleration indicated by the acceleration data Db1 and the angularvelocity indicated by the angular velocity data Db2, to thereby updatethe current direction data Dd2 using data indicating the currentdirection such that the calculated x-axis, y-axis, and z-axis directionsare the current direction.

Here, the CPU 10 can calculate the amount of rotation (the amount ofchange in the direction) of the terminal apparatus 6 in real space perunit time, using the angular velocity indicated by the angular velocitydata Db2. Further, in the state where the terminal apparatus 6 issubstantially stationary (in a static state) in real space, theacceleration applied to the terminal apparatus 6 is the gravitationalacceleration. This makes it possible to calculate the direction ofgravity applied to the terminal apparatus 6 (i.e., the attitude of theterminal apparatus 6 with respect to the vertical direction in realspace, and the x-axis, y-axis, and z-axis directions with respect to thevertical direction), using the acceleration indicated by theacceleration data Db1. This enables the CPU 10 to calculate a change inthe direction and the attitude of the terminal apparatus 6 on the basisof the acceleration indicated by the acceleration data Db1 and theangular velocity indicated by the angular velocity data Db2.

It should be noted that in the exemplary embodiment, a change in thedirection and the attitude of the terminal apparatus 6 are calculated onthe basis of the data indicating the acceleration and the angularvelocity that are detected by the terminal apparatus 6. Alternatively,in another embodiment, a change in the direction and the attitude of theterminal apparatus 6 may be calculated using any one piece of data orthree or more pieces of data. For example, the magnetic sensor 602included in the terminal apparatus 6 detects a geomagnetism applied tothe terminal apparatus 6. This makes it possible to calculate apredetermined orientation with respect to the terminal apparatus 6(i.e., the attitude of the terminal apparatus 6 with respect to thepredetermined orientation) on the basis of the direction of thegeomagnetism applied to the terminal apparatus 6. Even when a magneticfield is generated in addition to the geomagnetism in the real spacewhere the terminal apparatus 6 is located, it is possible to calculatethe amount of rotation of the terminal apparatus 6. This enables the CPU10 to calculate a change in the direction and the attitude of theterminal apparatus 6 using at least one of the data indicating theacceleration, the data indicating the angular velocity, and the dataindicating the magnetism, which are detected by the terminal apparatus6.

Any calculation method may be used to calculate the attitude of theterminal apparatus 6. For example, a calculation method is possibly usedof correcting the attitude of the terminal apparatus 6, which iscalculated on the basis of the angular velocity indicated by the angularvelocity data Db2, using the acceleration indicated by the accelerationdata Db1 and the direction of the magnetism detected by the magneticsensor 602.

Specifically, the CPU 10 first calculates the attitude of the terminalapparatus 6 on the basis of the angular velocity indicated by theangular velocity data Db2. Any method may be used to calculate theattitude of the terminal apparatus 6 from the angular velocity. Forexample, the attitude of the terminal apparatus 6 may be calculatedusing the most recent attitude (the most recently calculated x-axis,y-axis, and z-axis directions) and the current angular velocity (theangular velocity currently acquired in step 42 in the processing loop).The CPU 10 causes the most recent x-axis, y-axis, and z-axis directionsto rotate about the axes along the respective directions at the currentangular velocity for a unit time, to thereby calculate new x-axis,y-axis, and z-axis directions. It should be noted that the most recentx-axis, y-axis, and z-axis directions are represented by the currentdirection data Dd2, and the current angular velocity is represented bythe angular velocity data Db2. Accordingly, the CPU 10 reads the currentdirection data Dd2 and the angular velocity data Db2, and calculates theattitude of the terminal apparatus 6 (new x-axis, y-axis, and z-axisdirections). It should be noted that, as described above, the initialattitude of the terminal apparatus 6 is defined in step 41 describedabove. Thus, when the attitude of the terminal apparatus 6 is calculatedfrom the angular velocity, the CPU 10 can calculate the current attitudeof the terminal apparatus 6 with respect to the initial attitude of theterminal apparatus 6 that has been calculated first.

Next, the CPU 10 corrects the attitude of the terminal apparatus 6 (thex-axis, y-axis, and z-axis directions), calculated on the basis of theangular velocity, using the acceleration indicated by the accelerationdata Db1. Specifically, the CPU 10 calculates the attitude of theterminal apparatus 6 (the x-axis, y-axis, and z-axis directions) on thebasis of the acceleration indicated by the acceleration data Db1. Here,in the state where the terminal apparatus 6 is substantially stationary,the acceleration applied to the terminal apparatus 6 is thegravitational acceleration. Accordingly, in this state, it is possibleto calculate the direction of the gravitational acceleration (thedirection of gravity) using the direction of the acceleration indicatedby the acceleration data Db1. This makes it possible to calculate thedirection of the terminal apparatus 6 relative to the direction ofgravity (the x-axis, y-axis, and z-axis directions with respect to thedirection of gravity).

When the attitude of the terminal apparatus 6 based on the accelerationis calculated, the CPU 10 corrects the attitude based on the angularvelocity, using the attitude based on the acceleration. Specifically,the CPU 10 makes a correction to approximate at a predetermined rate theattitude of the terminal apparatus 6 (the x-axis, y-axis, and z-axisdirections) calculated on the basis of the angular velocity to theattitude of the terminal apparatus 6 (the x-axis, y-axis, and z-axisdirections) calculated on the basis of the acceleration. Thepredetermined rate may be a fixed value set in advance, or may be set inaccordance with, for example, the acceleration indicated by theacceleration data Db1. Further, the attitude of the terminal apparatus 6calculated on the basis of the acceleration cannot be calculated in thedirection of rotation about the direction of gravity, and therefore, theCPU 10 may not make a correction on the attitude in this rotationdirection. When correcting, on the basis of the direction of magnetismdetected by the magnetic sensor 602, the attitude of the terminalapparatus 6 calculated on the basis of the angular velocity, the CPU 10may approximate at a predetermined rate the attitude of the terminalapparatus 6 calculated on the basis of the angular velocity to theattitude of the terminal apparatus 6 calculated on the basis of thedirection of magnetism detected by the magnetic sensor 602. This enablesthe CPU 10 to accurately calculate the attitude of the terminalapparatus 6.

Next, the CPU 10 performs a player object setting process (step 83), andproceeds to the subsequent step. With reference to FIG. 19, adescription is given below of the player object setting process in step83 described above.

Referring to FIG. 19, the CPU 10 performs an operation indicationdirection calculation process (step 121), and proceeds to the subsequentstep. With reference to FIG. 20, a description is given below of theoperation indication direction calculation process performed in step 121described above.

Referring to FIG. 20, the CPU 10 corrects the up-down and forwarddirections of the terminal apparatus 6 (step 221), and proceeds to thesubsequent step. For example, the CPU 10 corrects the direction(attitude) of the terminal apparatus 6 such that the horizontaldirection is indicated by the state of the terminal apparatus 6 beingdirected downward by a predetermined angle (e.g., 20°) relative to thehorizontal direction in real space. Specifically, in step 82 describedabove, the CPU 10 calculates the x-axis, y-axis, and z-axis directionsof the terminal apparatus 6 in real space on the basis of theacceleration indicated by the acceleration data Db1 and the angularvelocity indicated by the angular velocity data Db2, to thereby updatethe current direction data Dd2 such that the calculated x-axis, y-axis,and z-axis directions are the current direction. In step 221 describedabove, the CPU 10 corrects the y-axis direction and the z-axisdirection, using all the x-axis, y-axis, and z-axis directions indicatedby the current direction data Dd2, such that the y-axis direction andthe z-axis direction are directed upward by the predetermined angleabout the x-axis direction (i.e., as viewed in the x-axis positivedirection, the y-axis direction and the z-axis direction are rotated tothe right by the predetermined angle about the x-axis).

Next, the CPU 10 corrects the tilt about the z-axis (step 222), andproceeds to the subsequent step. For example, the CPU 10 corrects thedirection (attitude) of the terminal apparatus 6 such that the x-axis ofthe terminal apparatus 6 is the horizontal direction in real space.Specifically, the CPU 10 rotates the x-axis direction about the z-axisdirection using the x-axis, y-axis, and z-axis directions corrected instep 221 described above, to thereby forcibly correct the x-axisdirection to the horizontal direction in the real space. Then, the CPU10 newly calculates the z-axis direction on the basis of the exteriorproduct of the corrected x-axis direction and the y-axis direction.Then, the CPU 10 newly calculates the y-axis direction on the basis ofthe exterior product of the newly calculated z-axis direction and thex-axis direction corrected to the horizontal direction, to therebyupdate the current direction data Dd2 using the newly calculated x-axis,y-axis, and z-axis directions.

It should be noted that also in the setting process of the referencedirection in step 41 described above, the x-axis, y-axis, and z-axisdirections of the terminal apparatus 6 are corrected as in steps 221 and222 described above, whereby the real space reference direction data Dd1is updated using the corrected z-axis positive direction as the realspace reference direction.

Next, the CPU 10 calculates the difference in horizontal angle betweenthe real space reference direction and the current direction (step 223),and proceeds to the subsequent step. Here, the difference in horizontalangle described above is the difference in angle obtained by projectingonto a horizontal plane the difference in angle between the real spacereference direction in real space (the initially set z-axis positivedirection) and the z-axis positive direction indicated by the currentdirection data Dd2. The difference in horizontal angle described aboveindicates the angle by which the direction of the terminal apparatus 6has changed from the initial attitude of the terminal apparatus 6 withrespect to the vertical direction in real space (the direction in whichthe back surface of the terminal apparatus 6 is directed (the z-axispositive direction shown in FIG. 3)). For example, the CPU 10 calculatesthe difference in horizontal angle described above, using the real spacereference direction indicated by the real space reference direction dataDd1 and the z-axis positive direction indicated by the current directiondata Dd2.

Next, the CPU 10 calculates the difference in up-down angle between thehorizontal direction in real space and the current direction (step 224),and proceeds to the subsequent step. For example, the CPU 10 calculates,as the difference in up-down angle, the difference in angle between thehorizontal direction in real space and the z-axis positive direction,using the z-axis positive direction indicated by the current directiondata Dd2.

Next, the CPU 10 calculates the operation indication direction relativeto the virtual world reference direction, in accordance with thedifference in horizontal angle calculated in step 223 described aboveand the difference in up-down angle calculated in step 224 describedabove (step 225), and ends the process of this subroutine. For example,the CPU 10 calculates the operation indication direction in the virtualworld, using the virtual world reference direction indicated by thevirtual world reference direction data De1, such that the difference inangle obtained by projecting the virtual world reference direction andthe operation indication direction onto a horizontal plane in thevirtual world is the difference in horizontal angle described above, andthe virtual world reference direction and the operation indicationdirection have the same positional relationship (i.e., the positionalrelationships are such that when the z-axis positive direction hasrotated to the left relative to the real space reference direction, alsothe operation indication direction rotates to the left relative to thevirtual world reference direction). Further, the CPU 10 calculates theoperation indication direction in the virtual world such that thedifference in angle between the horizontal direction in the virtualworld and the operation indication direction is the difference inup-down angle described above, and the horizontal direction in thevirtual world and the operation indication direction have the samepositional relationship (i.e., when the z-axis positive direction isdirected downward relative to the horizontal direction in real space,also the operation indication direction is directed downward relative tothe horizontal direction in the virtual world). Then, the CPU 10 updatesthe operation indication direction data De2 using the calculatedoperation indication direction.

Referring back to FIG. 19, after the operation indication directioncalculation process in step 121 described above, the CPU 10 determineswhether or not the operation indication direction is included in thebarrel left-right operation range (step 122). When the operationindication direction is included in the barrel left-right operationrange, the CPU 10 proceeds to the subsequent step 123. On the otherhand, when the operation indication direction is not included in thebarrel left-right operation range, the CPU 10 proceeds to the subsequentstep 124. Here, as described with reference to FIG. 14A, the barrelleft-right operation range is the range where the direction of thebarrel of the water cannon is allowed to be changed to the left andright (in the horizontal direction) in accordance with the operationindication direction, and the barrel left-right operation range is setto a predetermined angular range about the virtual world referencedirection. Then, in step 122 described above, the CPU 10 determines,using the virtual world reference direction indicated by the virtualworld reference direction data De1 and the operation indicationdirection indicated by the operation indication direction data De2,whether or not the difference in angle obtained by projecting thevirtual world reference direction and the operation indication directiononto a horizontal plane in the virtual world is included in the barrelleft-right operation range.

In step 123, the CPU 10 sets the left-right direction of the barrel andthe left-right direction of the first virtual camera in accordance withthe operation indication direction calculated in step 121 describedabove, and proceeds to the subsequent step 126. For example, the CPU 10sets the direction, obtained by projecting the operation indicationdirection indicated by the operation indication direction data De2 ontoa horizontal plane in the virtual world, as it is to the left-rightdirection of the barrel and the left-right direction of the firstvirtual camera, to thereby update the barrel left-right direction dataDf1 and the data concerning the left-right direction of the firstvirtual camera in the virtual camera data Di, using the set left-rightdirection of the barrel and the set left-right direction of the firstvirtual camera. It should be noted that in the form where the playerobject Po moves in the virtual world on the basis of the attitude andthe motion of the terminal apparatus 6, a moving angle and a movingdistance may be calculated on the basis of the set left-right directionof the barrel, and the position of the player object Po in the virtualworld may be calculated in accordance with the moving angle and themoving distance, to thereby newly set data concerning the calculatedposition as well.

On the other hand, in step 124, the CPU 10 sets the left-right directionof the barrel so as to be limited in the barrel left-right operationrange, and proceeds to the subsequent step. For example, the CPU 10 setsthe left-right direction of the barrel in the barrel left-rightoperation range so as to be closest to the direction obtained byprojecting the operation indication direction indicated by the operationindication direction data De2 onto a horizontal plane in the virtualworld, to thereby update the barrel left-right direction data Df1 usingthe set left-right direction of the barrel. It should be noted that evenin the form where the player object Po moves in the virtual world on thebasis of the attitude and the motion of the terminal apparatus 6, if theleft-right direction of the barrel that is limited does not change fromthe most recently set left-right direction of the barrel, the playerobject Po is not caused to move in the virtual world, and the dataconcerning the position is not updated from the most recent setting.

Next, the CPU 10 sets the left-right direction of the first virtualcamera in accordance with the operation indication direction calculatedin step 121 described above (step 125), and proceeds to the subsequentstep 126. For example, the CPU 10 sets the direction, obtained byprojecting the operation indication direction indicated by the operationindication direction data De2 onto a horizontal plane in the virtualworld, as it is to the left-right direction of the first virtual camera,to thereby update the data concerning the left-right direction of thefirst virtual camera in the virtual camera data Di, using the setleft-right direction of the first virtual camera.

In step 126, the CPU 10 determines whether or not the operationindication direction is included in the barrel up-down operation range.When the operation indication direction is included in the barrelup-down operation range, the CPU 10 proceeds to the subsequent step 127.On the other hand, when the operation indication direction is notincluded in the barrel up-down operation range, the CPU 10 proceeds tothe subsequent step 128. Here, as described with reference to FIG. 14B,the barrel up-down operation range is the range where the direction ofthe barrel of the water cannon is allowed to be changed upward anddownward (in the vertical direction) in accordance with the operationindication direction, and the barrel up-down operation range is set to apredetermined angular range with respect to the horizontal direction inthe virtual world. Then, in step 126 described above, the CPU 10determines, using the operation indication direction indicated by theoperation indication direction data De2, whether or not the differencein angle between the horizontal direction in the virtual world and theoperation indication direction is included in the barrel up-downoperation range.

In step 127, the CPU 10 sets the up-down direction of the barrel and theup-down direction of the first virtual camera in accordance with theoperation indication direction calculated in step 121 described above,and proceeds to the subsequent step 130. For example, the CPU 10 setsthe direction, obtained by projecting the operation indication directionindicated by the operation indication direction data De2 onto a verticalplane in the virtual world, as it is to the up-down direction of thebarrel and the up-down direction of the first virtual camera, to therebyupdate the barrel up-down direction data Df2 and the data concerning theup-down direction of the first virtual camera in the virtual camera dataDi, using the set up-down direction of the barrel and the set up-downdirection of the first virtual camera. It should be noted that in theform where the player object Po moves also upward and downward in thevirtual world on the basis of the attitude and the motion of theterminal apparatus 6, a moving angle and a moving distance may becalculated on the basis of the set up-down direction of the barrel, andthe position of the player object Po in the virtual world may becalculated in accordance with the moving angle and the moving distance,to thereby newly set data concerning the calculated position as well.

On the other hand, in step 128, the CPU 10 sets the up-down direction ofthe barrel so as to be limited in the barrel up-down operation range,and proceeds to the subsequent step. For example, the CPU 10 sets theup-down direction of the barrel in the barrel up-down operation range soas to be closest to the direction obtained by projecting the operationindication direction indicated by the operation indication directiondata De2 onto a vertical plane in the virtual world, to thereby updatethe barrel up-down direction data Df2 using the set up-down direction ofthe barrel. It should be noted that even in the form where the playerobject Po moves also upward and downward in the virtual world on thebasis of the attitude and the motion of the terminal apparatus 6, if theup-down direction of the barrel that is limited does not change from themost recently set up-down direction of the barrel, the player object Pois not caused to move in the virtual world, and the data concerning theposition is not updated from the most recent setting.

Next, the CPU 10 sets the up-down direction of the first virtual camerain accordance with the operation indication direction calculated in step121 described above (step 129), and proceeds to the subsequent step 130.For example, the CPU 10 sets the direction, obtained by projecting theoperation indication direction indicated by the operation indicationdirection data De2 onto a vertical plane in the virtual world, as it isto the up-down direction of the first virtual camera, to thereby updatethe data concerning the up-down direction of the first virtual camera inthe virtual camera data Di, using the set up-down direction of the firstvirtual camera.

In step 130, the CPU 10 performs a discharge object setting process, andproceeds to the subsequent step. With reference to FIG. 21, adescription is given below of the discharge object setting processperformed in step 130 described above.

In FIG. 21, the CPU 10 determines whether or not the discharge of thedischarge object is present (step 231). When the discharge of thedischarge object is present, the CPU 10 proceeds to the subsequent step232. On the other hand, when the discharge of the discharge object isnot present, the CPU 10 ends the process of this subroutine. Here, theplayer object Po can discharge the discharge object such as the water W,using the water cannon in operation, and the discharge object isdischarged from the barrel in the set direction of the barrel inaccordance with a predetermined operation of the user (e.g., theoperation of applying to the board-type controller 9 a load equal to orgreater than a predetermined threshold). The state, determined in step231 described above, where “the discharge of the discharge object ispresent” indicates the case where the predetermined operation is beingperformed (the case where the latest total load value indicated by theload value data Dc is equal to or greater than the predeterminedthreshold) and/or the case where the discharge object is discharged andmoving in the virtual world (a discharge vector whose magnitude is otherthan 0 is set in the discharge vector data Dg3).

In step 232, the CPU 10 determines whether or not the total load valueis equal to or greater than a predetermined threshold. Here, thethreshold used in step 232 described above is a value set in advance fordetermining whether or not the user is performing a discharge operationusing the board-type controller 9. When the value of the latest totalload applied to the board-type controller 9 has become equal to orgreater than the threshold, it is determined that the dischargeoperation is being performed. When, with reference to the latest totalload value indicated by the load value data Dc, the total load value isequal to or greater than the predetermined threshold, the CPU 10proceeds to the subsequent step 233. On the other hand, when the latesttotal load value is less than the predetermined threshold, the CPU 10proceeds to the subsequent step 234.

In step 233, on the basis of the total load value and the direction ofthe barrel, the CPU 10 adds the data concerning the discharge object(the type of the discharge object, the amount of discharge, and thedischarge vector), and proceeds to the subsequent step 234. For example,when the discharge operation of discharging the discharge object hasbeen performed, the CPU 10 sets, in the barrel of the water cannon, theposition of the discharge object to be newly discharged, and also setsthe direction of the barrel of the water cannon to the direction of thedischarge vector of the discharge object. Further, in accordance withthe latest total load value indicated by the load value data Dc, the CPU10 calculates the discharge velocity and the amount of discharge perunit time at and in which the discharge object is to be newlydischarged. Specifically, the CPU 10 sets the discharge velocity and theamount of discharge per unit time such that the greater the latest totalload value, the greater the discharge velocity, and the greater theamount of discharge per unit time. Further, with reference to thehistory of the total load value indicated by the load value data Dc, theCPU 10 sets a first discharge object (e.g., the water W) as thedischarge object to be newly discharged when the amount of change fromthe total load value calculated in the most recent processing to thelatest total load value is less than a predetermined value. The CPU 10sets a second discharge object (e.g., the large ball formed of a mass ofwater having a greater amount than that of the water W) as the dischargeobject to be newly discharged when the amount of change is equal to orgreater than the predetermined value. Then, the CPU 10 adds the objecttype data Dg1, the amount of discharge data Dg2, the discharge vectordata Dg3, and the position data Dg4 that indicate the type, the amountof discharge, the discharge vector, and the position of the setdischarge object, to the discharge object data Dg as data concerning newdischarge object. Step 233 described above is thus repeated, wherebydata concerning new discharge object is added to the discharge objectdata Dg.

It should be noted that step 233 described above is repeated in eachprocessing cycle of the game apparatus body 5 when the latest total loadvalue is equal to or greater than the predetermined threshold. In thiscase, a new discharge object is generated in each cycle. Accordingly,the cycle of repeating step 233 described above may be appropriately setin accordance with the times at which it is desired to generate a newdischarge object in the virtual world. In this case, the process of step232 described above is performed at the desired times, and the processesof steps 232 and 233 described above are not performed except at thedesired times.

In step 234, the CPU 10 causes the discharge objects set in thedischarge object data Dg to move on the basis of the respectivedischarge vectors, and ends the process of the subroutine. For example,on the basis of the discharge vectors set in the discharge object dataDg, the CPU 10 causes the discharge objects to move in the virtualworld, and sets new positions of the discharge objects, to therebyupdate the position data Dg4 of the discharge objects using the setpositions. Further, on the basis of the environment of the virtual world(the force of gravity, wind, the effects of other objects, and the like)where the discharge objects are placed, the CPU 10 corrects thedischarge vectors of the discharge objects, to thereby update thedischarge vector data Dg3 of the discharge objects using the correcteddischarge vectors. It should be noted that when any of the dischargeobjects collides with another object due to the above movements, the CPU10 sets data indicating that the discharge object has collided with saidanother object, and also deletes the data concerning the dischargeobject (the object type data Dg1, the amount of discharge data Dg2, thedischarge vector data Dg3, and the position data Dg4) from the dischargeobject data Dg.

Step 233 described above is thus repeated, whereby new discharge objectsare repeatedly set in the barrel of the water cannon operated by theplayer object Po, and also a discharge vector whose discharge directionis the direction of the barrel is set for each of the newly setdischarge objects. Then, step 234 described above is repeated, wherebythe positions of the discharge objects set in the discharge object dataDg are set so as to move in the virtual world on the basis of thedischarge vectors correspondingly set for the discharge objects. Forexample, as shown in FIG. 23, discharge objects W1 through W15 movesuccessively in the virtual world on the basis of discharge vectors Vw1through Vw15 whose vector directions are each the direction of thebarrel when the discharge object is discharged, and whose magnitudes areeach the discharge velocity based on the total load value when thedischarge object is discharged. Accordingly, when the user has performedthe operation of changing the direction of the barrel while performingthe discharge operation, discharge vectors are set so as to havedirections different between the discharge objects discharged from thebarrel. This results in causing the discharge objects W1 through W15 tomove successively in a meandering manner in the virtual world. Here, thepositions to be reached by the discharge objects in the virtual worldare determined in an analog manner on the basis of the direction of thebarrel at the time of the discharge and the discharge velocity based onthe total load value at the time of the discharge. This makes itdifficult for the user playing the game to predict the positions to bereached by the discharge objects. When, however, having consecutivelydischarged discharge objects, the user may cause the discharge objectsto move successively, and thereby can predict, with reference to thepositions reached by the previously discharged discharge objects, thepositions to be reached by the discharge objects to be dischargedthereafter.

Referring back to FIG. 19, after the discharge object setting process instep 130 described above, the CPU 10 performs an attack receptionoperation (step 131), and ends the process of this subroutine. Withreference to FIG. 22, a description is given below of the attackreception operation performed in step 131 described above.

In FIG. 22, the CPU 10 determines whether or not an enemy bomb object Bmoving in the virtual world is present (step 241). When an enemy bombobject B moving in the virtual world is present, the CPU 10 proceeds tothe subsequent step 242. On the other hand, when an enemy bomb object Bmoving in the virtual world is not present, the CPU 10 proceeds to thesubsequent step 246. Here, an enemy object Eo placed in the virtualworld may throw an enemy bomb object B at the player object Po at randomtiming (see FIG. 10A). The determination that “an enemy bomb object B ispresent” in step 241 described above indicates the case where an enemyobject Eo has taken the action of throwing an enemy bomb object B torelease the enemy bomb object B into the virtual world and/or the casewhere an enemy bomb object B thrown by an enemy object Eo is moving inthe virtual world (the case where a movement vector whose magnitude isother than 0 is set in the enemy bomb object data Dh).

In step 242, the CPU 10 causes the enemy bomb objects B set in the enemybomb object data Dh to move on the basis of the respective movementvectors, and proceeds to the subsequent step. For example, on the basisof the movement vectors set in the enemy bomb object data Dh, the CPU 10causes the respective enemy bomb objects B to move in the virtual world,and sets new positions of the enemy bomb objects B, to thereby updateposition data of the enemy bomb objects B using the set positions.Further, on the basis of the environment (the effects of the force ofgravity, wind, and other objects) of the virtual world where the enemybomb objects B are placed, the CPU 10 corrects the movement vectors ofthe enemy bomb objects B, to thereby update movement vector data of theenemy bomb objects B using the corrected movement vectors.

It should be noted that when an enemy object Eo has taken the action ofthrowing an enemy bomb object B to release the enemy bomb object B intothe virtual world, the CPU 10, in step 242 described above, adds dataconcerning the enemy bomb object B (the type, the size, the position,and the movement vector of the enemy bomb object B) to the enemy bombobject data Dh on the basis of a predetermined algorithm. For example,when an enemy object Eo has released an enemy bomb object B, theposition of the enemy bomb object B to be newly thrown is set at theposition of the enemy object Eo (to be exact, the position of the end ofthe arm used to throw the enemy bomb object B), and the type, the size,and the movement vector of the enemy bomb object B are set on the basisof a predetermined algorithm. Then, the CPU 10 adds to the enemy bombobject data Dh the type, the size, the position, and the movement vectorof the enemy bomb object B that have been set.

Next, the CPU 10 performs a process of collision detection between eachof the enemy bomb objects B moving in the virtual world and anotherobject (step 243), and proceeds to the subsequent step. As an example,the CPU 10 sets a contact determination area (e.g., a collision area) onthe player object Po and each of the enemy bomb objects B, and makescollision detection between the player object Po and each of the enemybomb objects B on the basis of the result of the contact determinationbetween the contact determination areas. Here, on the basis of theleft-right direction of the barrel and the up-down direction of thebarrel (i.e., the barrel direction data Df) that have been set in theprocess of step 123, 124, 127, or 128 described above, the CPU 10determines the direction, in the virtual world, of the collision area tobe set on the player object Po. Further, in the form where the playerobject Po moves in the virtual world on the basis of the attitude andthe motion of the terminal apparatus 6, the CPU 10 determines theposition, in the virtual world, of the collision area to be set on theplayer object Po on the basis of the position of the player object Poset in the process of step 123, 124, 127, or 128 described above.Further, on the basis of the positions of the enemy bomb objects B setin step 242 described above, the CPU 10 determines the positions, in thevirtual world, of the collision areas to be set on the enemy bombobjects B. Collision detection between the player object Po and each ofthe enemy bomb objects B is made using the thus set collision areas.When any of the enemy bomb objects B has collided with another objectincluding the player object Po, the CPU 10 sets data indicating that theenemy bomb object B has collided with said another object, and deletesdata concerning the enemy bomb object B from the enemy bomb object dataDh.

Next, the CPU 10 determines whether or not the player object Po and anyof the enemy bomb objects B have collided with each other in thecollision detection process in step 243 described above (step 244). Whenthe player object Po and any of the enemy bomb objects B have collidedwith each other, the CPU 10 proceeds to the subsequent step 245. On theother hand, when the player object Po and none of the enemy bomb objectsB have collided with each other, the CPU 10 proceeds to the subsequentstep 246.

In step 245, the CPU 10 generates a dirt image in which dirt clumps Bdare represented so as to be attached to the surface of the LCD 61, andproceeds to the subsequent step 246. For example, the CPU 10 sets thenumber of dirt clumps Bd to be attached, the positions of the dirtclumps Bd, the sizes of the dirt clumps Bd, the shapes of the dirtclumps Bd, the colors of the dirt clumps Bd, the transparencies of thedirt clumps Bd, and the like on the basis of the type and the size ofthe enemy bomb object B having collided with the player object Po basedon the detection made in step 243 described above. Then, in accordancewith the set conditions of the dirt clumps Bd, the CPU 10 generates adirt image to be displayed on the LCD 61, to thereby update the dirtimage data Dj3 using the generated dirt image. It should be noted thatif data of a dirt image indicating that dirt clumps Bd are attached isalready set in the dirt image data Dj3, the CPU 10 generates a new dirtimage by drawing on the already set dirt image the dirt clumps Bd of thedirt image generated in step 245 described above, to thereby update thedirt image data Dj3 using the newly generated dirt image.

It should be noted that the process of attaching a dirt clump Bd to thesurface of the LCD 61 may be performed in accordance with the case wherean enemy bomb object B has collided with a specific portion of theplayer object Po, or in accordance with the type, the size, thevelocity, and the like of the enemy bomb object B having collided withthe player object Po. In the first case, a collision area correspondingto a specific portion of the player object Po (e.g., the facial surfaceor the front surface of the player object Po) is set. When the collisionarea has collided with the collision area on the enemy bomb object B, apositive determination is made in step 244 described above, and then,the process of step 245 described above is performed. In this case, evenwhen the player object Po has collided with the enemy bomb object B, adirt clump Bd may not be attached depending on the facing direction ofthe player object Po in the virtual world. Accordingly, even when theplayer object Po can only rotate without moving in the virtual world, itis possible to avoid the attachment of a dirt clump Bd on the basis ofthe direction in which the terminal apparatus 6 is directed. In thesecond case, a positive determination is made in step 244 describedabove: when the type of the enemy bomb object B having collided with theplayer object Po is a specific type; when the size of the enemy bombobject B having collided with the player object Po is equal to orgreater than a predetermined size; when the moving velocity of the enemybomb object B having collided with the player object Po is equal to orgreater than a predetermined velocity; or the like. Then, the process ofstep 245 described above is performed.

In step 246, the CPU 10 determines whether or not the touch operationhas been performed on the touch panel 62. For example, with reference tothe touch position data Db3, the CPU 10 determines whether or not theuser has performed the touch operation on the touch panel 62. When thetouch operation has been performed on the touch panel 62, the CPU 10proceeds to the subsequent step 247. On the other hand, when the touchoperation has not been performed on the touch panel 62, the CPU 10 endsthe process of this subroutine.

In step 247, the CPU 10 removes an area in the dirt clump Bd in theperiphery of the touch position from the dirt image, to thereby generatea new dirt image, and ends the process of the subroutine. For example,the CPU 10 sets as a removal target area a predetermined range whosecenter is the touch position indicated by the touch position data Db3.Then, the CPU 10 extracts an image area corresponding to the removaltarget area in the dirt image indicated by the dirt image data Dj3(e.g., an image area displayed so as to overlap the removal target areawhen the dirt image is displayed on the LCD 61), and, when a dirt clumpBd is present in the image area, removes an area in the dirt clump Bdfrom the image area. Then, the CPU 10 updates the dirt image data Dj3using the dirt image from which the area in the dirt clump Bd has beenremoved.

Referring back to FIG. 18, after the player object setting process instep 83 described above, the CPU 10 sets parameters concerning thesecond virtual camera (step 84), and ends the process of thissubroutine. For example, a terminal game image and a monitor game imageare generated as, for example, three-dimensional CG images obtained bycalculating a game space viewed from a virtual camera placed in thevirtual world. As an example, the first virtual camera for generating aterminal game image is placed at the position of the first-person pointof view of the player object Po in the virtual world, the position beingthe head of the player object Po. Then, the first virtual camera is setsuch that the direction based on the left-right direction of the firstvirtual camera and the up-down direction of the first virtual camerathat have been set in the processes of steps 123, 125, 127, and 129described above is the direction of the line of sight of the firstvirtual camera, and the width direction of the first virtual camera isthe horizontal direction in the virtual world. Further, the secondvirtual camera for generating a monitor game image is set in the samevirtual world where the first virtual camera is set, the second virtualcamera set in a fixed manner so as to include the state of the virtualworld obtained by viewing from a distant bird's-eye view the playerobject Po placed in the virtual world. A terminal game image and amonitor game image are game images of the virtual world that are thusviewed from different points of view. This causes the game images of thevirtual world viewed from the different points of view to be displayedon the LCD 61 and the monitor 2.

Referring back to FIG. 17, after the game control process in step 44,the CPU 10 and the GPU 32 generate a monitor game image to be displayedon the monitor 2 (step 45), and proceed to the subsequent step. Forexample, the CPU 10 and the GPU 32 read from the main memory the dataindicating the result of the game control process performed in step 44,and read from the VRAM 34 the data used to generate a monitor gameimage. Then, the CPU 10 and the GPU 32 generate a game image using theread data, and store the generated monitor game image in the VRAM 34.Any monitor game image may be generated by any method so long as themonitor game image represents the result of the game control processperformed in step 44. For example, the monitor game image may be athree-dimensional CG image generated by the steps of: placing the secondvirtual camera in the virtual world on the basis of the parametersconcerning the second virtual camera that are indicated by the virtualcamera data Di; placing in the virtual world the player object Po thatoperates the water cannon, on the basis of the barrel direction data Df;placing the discharge object in the virtual world on the basis of thedischarge object data Dg; and calculating the virtual world viewed fromthe second virtual camera. Specifically, the CPU 10 places the playerobject Po and the water cannon in the virtual world such that the barrelof the water cannon operated by the player object Po is directed in thedirection of the barrel indicated by the barrel direction data Df.Further, the CPU 10 determines, on the basis of the type and the amountof discharge of the object that are indicated by the discharge objectdata Dg, the type and the size of the discharge object to be placed, andplaces the discharge object, on which the determinations have been made,in the virtual world on the basis of the position indicated by thedischarge object data Dg.

Next, the CPU 10 and the GPU 32 generate a terminal game image to bedisplayed on the terminal apparatus 6 (step 46), and proceed to thesubsequent step. For example, the CPU 10 and the GPU 32 read from themain memory the data indicating the result of the game control processperformed in step 44, and read from the VRAM 34 the data used togenerate a terminal game image. Then, the CPU 10 and the GPU 32 generatea terminal game image using the read data, and store the generatedterminal game image in the VRAM 34. Similarly to the monitor game image,any terminal game image may be generated by any method so long as theterminal game image represents the result of the game control processperformed in step 44. Further, the terminal game image may be generatedby the same method as, or a different method from, that for the monitorgame image. For example, a virtual world image may be generated as athree-dimensional CG image by the steps of: placing the first virtualcamera in the virtual world on the basis of the parameters concerningthe first virtual camera that are indicated by the virtual camera dataDi; placing in the virtual world the player object Po that operates thewater cannon, on the basis of the barrel direction data Df; placing thedischarge object in the virtual world on the basis of the dischargeobject data Dg; and calculating the virtual world viewed from the firstvirtual camera. Then, the dirt image indicated by the dirt image dataDj3 is superimposed on the virtual world image such that the dirt imageis given preference, whereby a terminal game image is generated andstored in the VRAM 34.

Next, the CPU 10 generates a monitor game sound to be output to theloudspeakers 2 a of the monitor 2 (step 47), and proceeds to thesubsequent step. For example, the CPU 10 causes the DSP 33 to generate amonitor game sound to be output from the loudspeakers 2 a, in accordancewith the result of the game control process performed in step 44. As anexample, the CPU 10 causes the DSP 33 to generate a monitor game soundin which BGM or the like to be output from the monitor 2 is added to thevoices and the action sounds of the objects, sound effects, and the likethat are supposed to be heard on the basis of the position of the secondvirtual camera in the virtual world set in accordance with the result ofthe game control process in step 44.

Next, the CPU 10 generates a terminal game sound to be output to theloudspeakers 607 of the terminal apparatus 6 (step 48), and proceeds tothe subsequent step. For example, the CPU 10 causes the DSP 33 togenerate a terminal game sound to be output from the loudspeakers 607,in accordance with the result of the game control process performed instep 44. As an example, the CPU 10 causes the DSP 33 to generate aterminal game sound in which BGM or the like to be output from theterminal apparatus 6 is added to the voices and the action sounds of theobjects, sound effects, and the like that are supposed to be heard onthe basis of the position of the first virtual camera in the virtualworld set in accordance with the result of the game control process instep 44. The terminal game sound may be the same as, or different from,the monitor game sound. Alternatively, the terminal game sound may bepartially different from the monitor game sound (e.g., the terminal gamesound and the monitor game sound include the same BGM and differentsound effects). It should be noted that when the monitor game sound andthe terminal game sound are the same, the terminal game sound generationstep in step 48 may not need to be performed.

Next, the CPU 10 outputs the monitor game image and the monitor gamesound to the monitor 2 (step 49), and proceeds to the subsequent step.For example, the CPU 10 transmits to the AV-IC 15 the data of themonitor game image stored in the VRAM 34 and the data of the monitorgame sound generated by the DSP 33. In response to this, the AV-IC 15transmits the data of the monitor game image and the data of the monitorgame sound to the monitor 2 through the AV connector 16. This causes themonitor game image to be displayed on the monitor 2, and causes themonitor game sound to be output from the loudspeakers 2 a.

Next, the CPU 10 transmits the terminal game image and the terminal gamesound to the terminal apparatus 6 (step 50), and proceeds to thesubsequent step. For example, the CPU 10 transmits to the codec LSI 27the data of the terminal game image stored in the VRAM 34 and the dataof the terminal game sound generated by the DSP 33. The codec LSI 27performs a predetermined compression process on the transmitted data.The compressed data of the terminal game image and the compressed dataof the terminal game sound are transmitted from the codec LSI 27 to theterminal communication module 28, and then transmitted from the terminalcommunication module 28 to the terminal apparatus 6 via the antenna 29.The data of the terminal game image and the data of the terminal gamesound that have been transmitted from the game apparatus body 5 arereceived by the wireless module 610 of the terminal apparatus 6, and aresubjected to a predetermined decompression process by the codec LSI 606.Then, the decompressed data of the terminal game image is output to theLCD 61, and the decompressed data of the terminal game sound is outputto the sound IC 608. This causes the terminal game image to be displayedon the LCD 61, and causes the terminal game sound to be output from theloudspeakers 607.

Next, the CPU 10 determines whether or not the game is to be ended (step51). Conditions for ending the game may be, for example: that particularconditions have been satisfied so that the game is over, or the game iscompleted; or that the user has performed an operation for ending thegame. When the game is not to be ended, the CPU 10 returns to step 42and repeats the same processing. On the other hand, when the game is tobe ended, the CPU 10 ends the processing of the flow chart. Thereafter,the series of processes 42 through 51 is repeatedly performed until theCPU 10 determines in step 51 that the game is to be ended.

As described above, the processing described above makes it possible tocontrol the action of the player object Po on the basis of the attitudeand the motion of the terminal apparatus 6, and also makes it possibleto cause the player object Po to take the action of avoiding an attackfrom an enemy object Eo. On the other hand, when an attack from an enemyobject Eo has been received, the effect of the attack (a dirt clump Bd)is displayed on the display apparatus of the terminal apparatus 6. Itis, however, also possible to repair the effect of the attack byperforming the touch operation so as to overlap the display of theeffect. The action of the player object Po avoiding an attack is thuscontrolled on the basis of the attitude of the terminal apparatus 6,whereby it is possible to provide the user, performing this operation,with a feeling as if being in the virtual world. Further, when an attackhas been received, the effect of the attack is displayed on the displayapparatus of the terminal apparatus 6 that is used to avoid attacks.This also makes it possible to provide the user, operating the terminalapparatus 6, with a feeling as if having received an attack in thevirtual world. Further, on the terminal apparatus 6 displaying an imagerepresenting the effect of an attack, an operation may be performed soas to touch the image, whereby it is also possible to remove the effectof the attack. This also makes it possible to provide the user,operating the terminal apparatus 6, with a feeling as if actuallyrepairing the effect of the attack received in the virtual world. Theattack avoidance operation, the display when an attack has beenreceived, and the operation of repairing the effect of the receivedattack are thus performed successively, whereby it is possible toimprove virtual reality dramatically.

In addition, based on the processing described above, the operationusing the terminal apparatus 6 in the barrel left-right operation rangechanges the direction of the line of sight of the first virtual cameraand the direction of the barrel of the water cannon to the left andright in accordance with the attitude and the motion of the terminalapparatus 6 that are obtained by yawing the direction of the terminalapparatus 6 to the left and right. Further, also the operation using theterminal apparatus 6 in the barrel up-down operation range changes thedirection of the line of sight of the first virtual camera and thedirection of the barrel of the water cannon in accordance with theattitude and the motion of the terminal apparatus 6 that are obtained bypitching the direction of the terminal apparatus 6 upward and downward.Accordingly, the operations using the terminal apparatus 6 in the barrelleft-right operation range and the barrel up-down operation range changenot only the direction of the line of sight of the virtual camera forgenerating the virtual world to be displayed on the terminal apparatus6, but also the discharge direction in which the discharge object is tobe discharged in the virtual world. These operations lead to anoperation suitable for adjusting the position to be reached by thedischarge object, and an operation suitable for changing the displayrange of the display performed on the LCD 61. On the other hand, theoperations using the terminal apparatus 6 outside the barrel left-rightoperation range and outside the barrel up-down operation range changeonly the direction of the line of sight of the first virtual camera inaccordance with the attitude and the motion of the terminal apparatus 6that are obtained by yawing the direction of the terminal apparatus 6 tothe left and right, and the attitude and the motion of the terminalapparatus 6 that are obtained by pitching the direction of the terminalapparatus 6 upward and downward. Accordingly, the operations using theterminal apparatus 6 outside the barrel left-right operation range andoutside the barrel up-down operation range lead to an operation suitablefor changing only the display range of the display performed on the LCD61 with the discharge direction unchanged. At least the barrelleft-right operation range and the barrel up-down operation range arethus set, whereby the user can perform various operations on the basisof the attitude and the motion of one device.

It should be noted that in the above descriptions, settings are made forthe range for determining the attitude and the direction of the motionof the terminal apparatus 6 that are obtained by yawing the direction ofthe terminal apparatus 6 to the left and right (the barrel left-rightoperation range), and the range for determining the attitude and thedirection of the motion of the terminal apparatus 6 that are obtained bypitching the direction of the terminal apparatus 6 upward and downward(the barrel up-down operation range). Alternatively, one range may beused, or three or more ranges may be used, to determine the abovedirections. For example, when the direction of the barrel of the watercannon changes only to the left and right in the virtual world, only thebarrel left-right operation range may be set, and always only theattitude of the first virtual camera may be changed in accordance withthe attitude and the motion of the terminal apparatus 6 that areobtained by pitching the direction of the terminal apparatus 6 upwardand downward. Yet alternatively, a range may be further set in which thevelocity of the change in the direction of the barrel of the watercannon is relatively small (e.g., ranges adjacent to the left and rightsides of the barrel left-right operation range, or ranges adjacent tothe top and bottom sides of the barrel up-down operation range), and thedirection of the barrel may be controlled such that the velocity of thechange in the direction of the barrel changes in accordance with thedirection the terminal apparatus 6.

In the exemplary game described above, the virtual camera (first virtualcamera) for generating an image to be displayed on the LCD 61 iscontrolled (the position, the direction, and the attitude of the virtualcamera are controlled) on the basis of the attitude of the terminalapparatus 6. Such control makes it possible to provide the user with animage as if peeping at the virtual world through the LCD 61, and providethe user with a feeling as if being in the virtual world. Further, theoperation using the attitude of the terminal apparatus 6 enables theoperation of rotating the terminal apparatus 6 in two directions, suchas a left-right swing (yaw) about the vertical direction (e.g., aboutthe y-axis direction) and an upward and downward swing (pitch) about theleft-right horizontal direction (e.g., about the x-axis direction), andtherefore is suitable for controlling the virtual camera capable ofmaking a similar movement also in the virtual world. Thus, the attitudeof the virtual camera in the virtual world may be controlled so as tocoincide with the attitude of the terminal apparatus 6 in real space,whereby it is possible to provide an image as if peeping in thedirection desired by the user in the virtual world. It should be notedthat in addition to the operation of rotating the terminal apparatus 6in two directions described above, the virtual camera may be controlledso as to rotate about the direction of the line of sight in accordancewith a left-right rotation (roll) about the front-back horizontaldirection (e.g., about the z-axis direction). The addition of suchcontrol enables the operation of rotating the terminal apparatus 6 inthree directions. Thus, the attitude of the virtual camera in thevirtual world may be controlled so as to coincide with the attitude ofthe terminal apparatus 6 in real space, whereby it is possible toprovide an image as if peeping in the direction desired by the user inthe virtual world.

In addition, in the exemplary game described above, in accordance withthe user taking action on the board-type controller 9, the player objecttakes action (e.g., a discharging action). That is, the user isprovided, by an image displayed on the LCD 61, with a feeling as ifbeing in the virtual world, and is additionally provided with anoperation feeling as if the user themselves is a player character inreal space. This enhances the feeling as if being in the virtual world.

In addition, in the exemplary game described above, the direction of thebarrel of the water cannon operated by the player object Po displayed onthe LCD 61 is controlled on the basis of the attitude of the terminalapparatus 6. Such control makes it possible to provide the user with anoperation environment as if the terminal apparatus 6 is a water cannon,and also provide a feeling as if the user is the player object Po in thevirtual world. Further, the operation using the attitude of the terminalapparatus 6 enables the operation of rotating the terminal apparatus 6in two directions, such as a left-right swing (yaw) about the verticaldirection (e.g., about the y-axis direction) and an upward and downwardswing (pitch) about the left-right horizontal direction (e.g., about thex-axis direction), and therefore is suitable for controlling the playerobject Po capable of making a similar movement also in the virtualworld. For example, in the exemplary game, a left-right swing (yaw)about the height direction along the LCD 61 of the terminal apparatus 6(the y-axis direction) may be set to correspond to a left-right change(yaw) in the direction of the barrel, and an upward and downward swing(pitch) about the left-right direction along the LCD 61 (the x-axisdirection) may be set to correspond to an upward and downward change(pitch) in the direction of the barrel, whereby it is possible toprovide a shooting game of changing the direction of the barrel to thedirection desired by the user in the virtual world.

In addition, in the exemplary game described above, the action of theplayer object Po discharging the discharge object (the presence orabsence of the discharge of the discharge object, the amount ofdischarge and the discharge velocity in and at which the dischargeobject is to be discharged, and the type of the discharge object) iscontrolled in accordance with the load value (total load value) to beapplied to the board-type controller 9. That is, it is possible toperform a discharge process based on an analog operation performed onthe board-type controller 9. Accordingly, the user controls the actionof one player object Po using a plurality of devices (the terminalapparatus 6 and the board-type controller 9). This makes it possible toperform an unprecedented operation, and also makes it possible toperform an analog operation on the action of the player object Po.

In addition, in the exemplary game described above, it is possible toset the perspective direction in the virtual world displayed on the LCD61 of the terminal apparatus 6, as the direction of the barrel of thewater cannon operated by the player object Po. This enables the user toset the direction of the barrel on the basis of the attitude of theterminal apparatus 6. Further, the virtual world is displayed on the LCD61 such that the direction of the barrel is the perspective direction.This enables the operation of setting the direction of the barrel in anintuitive manner, which facilitates the setting of the direction of thebarrel to the direction desired by the user.

In addition, in the above descriptions, when an attack from an enemyobject Eo has been received, a dirt image in which a dirt clump Bd isdisplayed is shown as an example of an image to be displayed on the LCD61 of the terminal apparatus 6 so as to represent the effect of theattack. The dirt clump Bd is used as an example of an image representingthe damage of the player object Po resulting from the attack from theenemy object Eo. Alternatively, another type of image representing thedamage may be displayed on the LCD 61. For example, as an imagerepresenting the damage (an image representing the effect of theattack), another type of image may be used that is displayed in avirtual world image in superimposition thereon to thereby hinder thefield of view toward the virtual world displayed in the virtual worldimage. Specifically, the following may be used as an image representingthe damage of the attack: an image in which a part of a transparentmember or a semi-transparent member, such as glass, is broken; an imagein which a part of the member is deformed; an image in which a part ofthe member is melted; an image in which a part of the member isdiscolored; an image in which an enemy object or the like is stuck to apart of the member; an image in which the transparency of a part of themember is low; an image in which the brightness of the virtual worldviewed through a part of the member is low; an image in which thesaturation of the virtual world viewed through a part of the member islow; an image in which the color density of the virtual world viewedthrough a part of the member is low; an image in which the virtual worldviewed through a part of the member is defocused; an image in which thevirtual world viewed through a part of the member is distorted; or thelike. Then, the image may be superimposed on the virtual world imagesuch that the image is given preference, whereby a terminal game imageis generated and displayed on the LCD 61. In any of these cases, controlis performed such that the touch operation is performed on the touchpanel 62 so as to overlap a part of the member on which the effect ofthe attack is displayed, whereby the effect of the attack describedabove is repaired. This makes it possible to perform processing similarto the game processing described above.

In addition, in the above descriptions, as an example, an area in a dirtclump Bd displayed when an attack from an enemy object Eo has beenreceived is deleted from an area that overlaps a predetermined rangewhose center is the touch position on the touch panel 62, whereby theeffect of the attack is repaired. Alternatively, the area in the dirtclump Bd may be deleted by another method. For example, representationmay be made such that the dirt clump Bd is formed by pasting a pluralityof dirt pattern images, and the effect of the attack is repaired bydeleting each of the dirt pattern images as a unit of deletion. As anexample, a deletion process is performed on each dirt pattern image bydeleting a dirt pattern image that overlaps the touch position on thetouch panel 62, or by increasing the transparency of the dirt patternimage by a predetermined level. In the first case, the dirt patternimage is deleted by performing the touch operation once. In the secondcase, the dirt pattern image is deleted by performing the touchoperation multiple times. In either case, a dirt pattern image displayedso as to overlap the touch position on the touch panel 62, or a dirtpattern image displayed so as to overlap a predetermined range whosecenter is the touch position, is handled as an area in an attack effectimage to be repaired so as to overlap the predetermined range whosecenter is the touch position. Further, even when the user has notperformed the touch operation, a dirt clump Bd may be deleted whenpredetermined conditions are satisfied. For example, when apredetermined period has elapsed since a dirt clump Bd has beendisplayed, the dirt clump Bd may be gradually deleted by incrementallyincreasing the transparency of the dirt clump Bd.

In addition, in the above descriptions, as an example of the dischargeobject, the water W and the large ball formed of a mass of water havinga greater amount than the water W are used. Alternatively, another typeof object may be used as the discharge object. For example, the term“discharge object” used in the present specification is one thatrepresents an object to be discharged or shot by the player object Po tohit another object with it, and examples of the “discharge object” mayalso include flames, bullets, shells, bombs, grenades, rockets,missiles, balls, arrows, beams, and laser beams in the virtual gameworld.

In addition, in the exemplary game described above, a virtual worldimage viewed from the first-person point of view of the player object Pois displayed on the LCD 61. Alternatively, an image of the virtual worldin another form may be displayed on the LCD 61. For example, an image ofthe virtual world including at least the player object Po may bedisplayed on the LCD 61 of the terminal apparatus 6. As an example, itis possible to place the first virtual camera at a position behind andclose to the player object Po, and display on the LCD 61 of the terminalapparatus 6 an image of the virtual world including at least the playerobject Po. Even in the exemplary game described above, only the attitudeof the first virtual camera may possibly change in the state where theaction of the player object Po is stopped because, outside the barrelleft-right operation range and outside the barrel up-down operationrange, the direction of the player object Po and the direction of thebarrel are locked in each range. In this case, the direction of the lineof sight of the first virtual camera may be set so as to view the partof the virtual world behind the player object Po such that the positionof the player object Po is the point of view. Alternatively, the firstvirtual camera may be caused to move to a position in front of and closeto the player object Po, or the first virtual camera may be set suchthat at least a part of the player object Po is included in the viewvolume.

In addition, in the exemplary game described above, when a virtual worldimage viewed from the first-person point of view of the player object Pois displayed on the LCD 61, a part of the player object Po (the barrelof the water cannon operated by the player object Po) is displayed.Alternatively, the player object Po may not be displayed at all. Forexample, even in the form where the player object Po is not displayed atall, an aim indicating the point to be reached by the discharge objectin the virtual world may be displayed on the LCD 61. Yet alternatively,as described above, when discharge objects have been discharged, thestate of the discharge objects being discharged successively indischarge order may be displayed.

It should be noted that in the exemplary game described above, theexemplary processing is performed such that in accordance with theoperation indication direction determined on the basis of the attitudeof the terminal apparatus 6, the action of the player object Po, thedirection of the barrel, and the attitude of the virtual camera arecontrolled in conjunction immediately after the determination. Inaccordance with the change in the operation indication direction,however, the action of the player object Po, the direction of thebarrel, and/or the attitude of the first virtual camera may becontrolled after a delay of a predetermined period. In this case, thevirtual world may be displayed such that: after the attitude of theplayer object Po and the direction of the barrel change, the attitude ofthe first virtual camera changes so as to follow the directions afterthe delay of the predetermined period; or after the attitude of thefirst virtual camera changes, the attitude of the player object Po andthe direction of the barrel change so as to follow the change in theattitude after the delay of the predetermined period.

In addition, in the exemplary game described above, the game imagedisplayed on the LCD 61 of the terminal apparatus 6 and the game imagedisplayed on the monitor 2 are images both representing the state of thesame virtual world, but are images different from each other in thepoint of view, and the range of view, toward the virtual world isviewed. This enables the user to view the virtual world displayed on thetwo display screens in different fields of view and different displayranges, and therefore enables the user to appropriately view a suitablegame image depending on the state of the game. Further, the exemplarygame described above enables the user to perform an operation whileholding the terminal apparatus 6, to thereby change the action of theplayer object Po and the direction of the barrel in accordance with theattitude and the position of the terminal apparatus 6 in real space, andalso change an image displayed on the LCD 61 in accordance with thechanges. This makes it possible to provide a sense of presence in thevirtual world to the user viewing an image displayed on the LCD 61 whileholding the terminal apparatus 6. On the other hand, viewing only animage displayed on the LCD 61 may make it difficult to understand theposition relative to the entire virtual world and the circumstance ofthe player object Po. The display of the virtual world in a relativelywide range on the monitor 2 can solve such a problem.

It should be noted that in the exemplary game described above, thesecond virtual camera for generating an image of the virtual world to bedisplayed on the monitor 2 is set in a fixed manner in the virtualworld. Alternatively, the position and the attitude of the secondvirtual camera may be changed in accordance with the motion of theplayer object Po. As an example, an image of the virtual world to bedisplayed on the monitor 2 may be generated by controlling the attitudeof the second virtual camera such that the direction in which thedirection of the line of sight of the second virtual camera is projectedonto a horizontal plane in the virtual world coincides with thedirection in which the direction of the barrel or the operationindication direction is projected onto the horizontal plane.

In addition, the game system 1 allows the user to perform various gamesusing the terminal apparatus 6 and the board-type controller 9 asoperation means. The terminal apparatus 6 can be used as a controllerthat allows the user to provide an input by an operation based on themotion of the body of the terminal apparatus 6, a touch operation, abutton operation, or the like, while it can be used as a portabledisplay or a second display. Accordingly, the game system 1 achieves awide range of games. That is, the terminal apparatus 6 functions as adisplay apparatus, and therefore, there may be a game system in which:the terminal apparatus 6 is used as display means while the monitor 2and the controller 7 are not used; and the board-type controller 9 isused as operation means. Further, the terminal apparatus 6 functions asan operation device as well as a display apparatus, and therefore, theremay be a game system in which the terminal apparatus 6 is used asdisplay means while the monitor 2 and the controller 7 are not used, andthe terminal apparatus 6 and the board-type controller 9 are used asoperation means. Further, the terminal apparatus 6 functions as anoperation device as well as a display apparatus, and therefore, theremay be a game system in which the terminal apparatus 6 is used asdisplay means while the monitor 2, the board-type controller 9, and thecontroller 7 are not used, and the terminal apparatus 6 is used asoperation means.

In addition, in the exemplary embodiment, the terminal apparatus 6functions as a so-called thin client terminal, which does not performgame processing. In the exemplary embodiment, however, at least a partof the series of steps in the game processing to be performed by thegame apparatus body 5 may be performed by the terminal apparatus 6. Asan example, the terminal game image generation process may be performedby the terminal apparatus 6. As another example, all the series of stepsin the game processing to be performed by the game apparatus body 5 maybe performed by the terminal apparatus 6. In this case, the terminalapparatus 6 functions as a processing device that performs the steps inthe game processing, as well as a display apparatus, and therefore,there may be a game system in which: the terminal apparatus 6 is used asdisplay means while the monitor 2, the game apparatus body 5, and thecontroller 7 are not used; the board-type controller 9 is used asoperation means; and the terminal apparatus 6 is used as processingmeans. In this game system, only the terminal apparatus 6 and theboard-type controller 9 are connected wirelessly or wired, and boardoperation data is transmitted from the board-type controller 9 to theterminal apparatus 6, thereby achieving various games. Further, it isneedless to say that when the board-type controller 9 is not usedeither, the terminal apparatus 6 may be used as display means, operationmeans, and processing means.

In addition, in the above embodiment, attitude data (e.g., at least onepiece of data output from the magnetic sensor 602, the accelerationsensor 603, and the gyro sensor 604) used to calculate the attitudeand/or the motion of the terminal apparatus 6 (including the positionand the attitude per se, or changes in the position and the attitude) isoutput from the terminal apparatus 6 to the game apparatus body 5, andthe attitude and/or the motion of the terminal apparatus 6 arecalculated by the information processing performed by the game apparatusbody 5. The attitude and/or the motion of the terminal apparatus 6 to becalculated by the game apparatus body 5, however, may be calculated bythe terminal apparatus 6. In this case, the data indicating the attitudeand/or the motion of the terminal apparatus 6 that have been calculatedby the terminal apparatus 6 (i.e., data indicating the position and theattitude per se of the terminal apparatus 6, or changes in the positionand the attitude that have been calculated using the attitude data) isoutput from the terminal apparatus 6 to the game apparatus body 5, andthe data is used in the information processing performed by the gameapparatus body 5.

In addition, in the above descriptions, the terminal apparatus 6 and thegame apparatus body 5 are connected by wireless communication, and theboard-type controller 9 and the game apparatus body 5 are connected bywireless communication. Alternatively, wireless communication betweendevices may be performed in a form other than the above. As a firstexample, the terminal apparatus 6 functions as a relay device foranother wireless communication. In this case, board operation data ofthe board-type controller 9 is wirelessly transmitted to the terminalapparatus 6, and the terminal apparatus 6 wirelessly transmits, to thegame apparatus body 5, terminal operation data of the terminal apparatus6 together with the received board operation data. In this case, whilethe terminal apparatus 6 and the game apparatus body 5 are directlyconnected by wireless communication, the board-type controller 9 isconnected to the game apparatus body 5 via the terminal apparatus 6 bywireless communication. As a second example, the board-type controller 9functions as a relay device for another wireless communication. In thiscase, terminal operation data of the terminal apparatus 6 is wirelesslytransmitted to the board-type controller 9, and the board-typecontroller 9 wirelessly transmits, to the game apparatus body 5, boardoperation data of the board-type controller 9 together with the receivedterminal operation data. In this case, the board-type controller 9 andthe game apparatus body 5 are directly connected by wirelesscommunication, while the terminal apparatus 6 is connected to the gameapparatus body 5 via the board-type controller 9 by wirelesscommunication.

In addition, the terminal apparatus 6 and/or the board-type controller 9may be electrically connected to the game apparatus body 5 via cables.In this case, the cables connected to the terminal apparatus 6 and/orthe board-type controller 9 are connected to a connection terminal ofthe game apparatus body 5. As a first example, the terminal apparatus 6and the game apparatus body 5 are electrically connected via a firstcable, and the board-type controller 9 and the game apparatus body 5 areelectrically connected via a second cable. As a second example, theterminal apparatus 6 and the game apparatus body 5 are electricallyconnected via a cable. In this case, board operation data of theboard-type controller 9 may be wirelessly transmitted to the terminalapparatus 6 and then transmitted to the game apparatus body 5 via thecable. As a third example, the board-type controller 9 and the gameapparatus body 5 are electrically connected via a cable. In this case,terminal operation data of the terminal apparatus 6 may be wirelesslytransmitted to the board-type controller 9 and then transmitted to thegame apparatus body 5 via the cable. Alternatively, terminal operationdata of the terminal apparatus 6 may be wirelessly transmitted to thegame apparatus body 5 directly from the terminal apparatus 6.

In addition, in the exemplary embodiment, the game system 1 includes oneterminal apparatus 6 and one board-type controller 9. Alternatively, thegame system 1 may be configured to include a plurality of terminalapparatuses 6 and a plurality of board-type controllers 9. That is, thegame apparatus body 5 may be capable of wirelessly communicating witheach terminal apparatus 6 and each type controller 9, and may transmitgame image data, game sound data, and control data to each terminalapparatus, and receive terminal operation data, camera image data,microphone sound data, and board operation data from each terminalapparatus 6 and each board-type controller 9. When the game apparatusbody 5 wirelessly communicates with the plurality of terminalapparatuses 6 and the plurality of board-type controllers 9, the gameapparatus body 5 may perform the wireless communication in a timedivision manner or in a frequency division manner.

As described above, when the game system 1 includes a plurality ofterminal apparatuses 6 and a plurality of board-type controllers 9, aplurality of users are allowed to play more games. For example, when thegame system 1 includes two pairs of terminal apparatuses 6 andboard-type controllers 9, two users are allowed to play a gamesimultaneously. Further, when the game system 1 includes two pairs ofterminal apparatuses 6 and board-type controllers 9, the game system 1includes three display apparatuses, and therefore can generate gameimages for three users to be displayed on the respective displayapparatuses.

In addition, in the above descriptions, a plurality of load sensors 94are provided in the board-type controller 9. Information of the positionof the center of gravity of a load applied to the board-type controller9, however, is not used in the above processing. Thus, at least one loadsensor 94 may be provided in the board-type controller 9.

In addition, the exemplary embodiment is described using the stationarygame apparatus 3. The exemplary embodiment, however, may be achieved byexecuting the game program according to the exemplary embodiment with aninformation processing apparatus such as a hand-held game apparatus or ageneral personal computer. Further, in another embodiment, the exemplaryembodiment may be applied not only to a game apparatus but also to agiven hand-held electronic device (e.g., a PDA (Personal DigitalAssistant) or a mobile telephone), a personal computer, a camera, andthe like. Any device may be connected to the terminal apparatus 6 andthe board-type controller 9 wirelessly or wired, whereby the exemplaryembodiment can be achieved.

In addition, in the above descriptions, the game processing is performedby the game apparatus body 5. At least a part of the processing steps inthe game processing, however, may be performed by another apparatusprovided outside the game system 1. For example, when the game apparatusbody 5 is configured to communicate with another apparatus (e.g., aserver or another game apparatus), the processing steps in the gameprocessing may be performed by the game apparatus body 5 in combinationwith said another apparatus. As an example, said another apparatusperforms the process of setting a player object, a virtual world, andthe like, and data concerning the motion and the attitude of the playerobject is transmitted from the game apparatus body 5 to said anotherapparatus, whereby the game processing is performed. Then, image dataindicating the virtual world generated by said other apparatus istransmitted to the game apparatus body 5, and the virtual world isdisplayed on the monitor 2 and the LCD 61. At least a part of theprocessing steps in the game processing is thus performed by anotherapparatus, whereby the same processing as the game processing isachieved. It should be noted that at least a part of the processingsteps in the information processing may be performed by the board-typecontroller 9 (the microcomputer 100). Further, the above game processingcan be performed by one processor or by a cooperation of a plurality ofprocessors, the one processor or the plurality of processors included inan information processing system including at least one informationprocessing apparatus. Further, in the exemplary embodiment, theprocesses shown in the above flow charts are performed as a result ofthe CPU 10 of the game apparatus body 5 executing a predeterminedprogram. Alternatively, a part or all of the processes may be performedby a dedicated circuit included in the game apparatus body 5.

The systems, devices and apparatuses described herein may include one ormore processors, which may be located in one place or distributed in avariety of places communicating via one or more networks. Suchprocessor(s) can, for example, use conventional 3D graphicstransformations, virtual camera and other techniques to provideappropriate images for display. By way of example and withoutlimitation, the processors can be any of: a processor that is part of oris a separate component co-located with the stationary display and whichcommunicates remotely (e.g., wirelessly) with the movable display; or aprocessor that is part of or is a separate component co-located with themovable display and communicates remotely (e.g., wirelessly) with thestationary display or associated equipment; or a distributed processingarrangement some of which is contained within the movable displayhousing and some of which is co-located with the stationary display, thedistributed portions communicating together via a connection such as awireless or wired network; or a processor(s) located remotely (e.g., inthe cloud) from both the stationary and movable displays andcommunicating with each of them via one or more network connections; orany combination or variation of the above.

The processors can be implemented using one or more general-purposeprocessors, one or more specialized graphics processors, or combinationsof these. These may be supplemented by specifically-designed ASICs(application specific integrated circuits) and/or logic circuitry. Inthe case of a distributed processor architecture or arrangement,appropriate data exchange and transmission protocols are used to providelow latency and maintain interactivity, as will be understood by thoseskilled in the art.

Similarly, program instructions, data and other information forimplementing the systems and methods described herein may be stored inone or more on-board and/or removable memory devices. Multiple memorydevices may be part of the same device or different devices, which areco-located or remotely located with respect to each other.

In addition, the shape of the game apparatus body 5 described above, theshapes of the terminal apparatus 6, the controller 7, and the board-typecontroller 9, and the shapes, the number, the placement, or the like ofthe various operation buttons and sensors are merely illustrative, andthe exemplary embodiment can be achieved with other shapes, numbers,placements, and the like. Further, the processing orders, the settingvalues, the display forms, the criterion values, and the like that areused in the information processing described above are also merelyillustrative, and it is needless to say that the exemplary embodimentcan be achieved with other orders, display forms, and values.

In addition, the game program described above may be supplied to thegame apparatus body 5 not only from an external storage medium such asthe optical disk 4, but also via a wireless or wired communication link.Further, the game program may be stored in advance in a nonvolatilestorage device of the game apparatus body 5. It should be noted thatexamples of an information storage medium for storing the game programmay include a CD-ROM, a DVD, given another optical disk storage mediumsimilar to these, a flexible disk, a hard disk, a magnetic optical disk,and a magnetic tape, as well as a nonvolatile memory. Furthermore, theinformation storage medium for storing the game program may be anonvolatile semiconductor memory or a volatile memory. Such storagemedia can be defined as storage media readable by a computer or thelike. For example, a computer or the like is caused to read and executeprograms stored in each of the storage media, and thereby can be causedto provide the various functions described above.

While some exemplary systems, exemplary methods, exemplary devices, andexemplary apparatuses have been described in detail, the foregoingdescription is in all aspects illustrative and not restrictive. It isunderstood that numerous other modifications and variations can bedevised without departing from the spirit and scope of the appendedclaims. It is understood that the scope of the exemplary embodimentshould be interpreted only by the appended claims. It is also understoodthat one skilled in the art can implement the exemplary embodiment inthe equivalent range on the basis of the description of the exemplaryembodiment and common technical knowledge, from the description of thespecific embodiments. It should be understood that when used in thepresent specification, components and the like described in singularform with the words “a” and “an” before them do not exclude theplurality of these components. Furthermore, it should be understood thatterms used in the present specification have meanings generally used inthe art unless otherwise specified. Therefore, unless otherwise defined,all the jargons and technical terms have the same meanings as thosegenerally understood by one skilled in the art of the exemplaryembodiment. In the event of any contradiction, the present specification(including meanings defined herein) has priority.

A storage medium having stored thereon a game program, a game apparatus,a game system, and a game processing method according to the exemplaryembodiment can improve virtual reality by causing a player object in avirtual world to take action in accordance with an operation of a user,and therefore are suitable for use as a game program, a game apparatus,a game system, and a game processing method that perform the process ofcausing a player object to take action on the basis of the operation andthe like.

What is claimed is:
 1. A computer-readable storage medium having storedthereon a game program to be executed by a computer of a game apparatuscapable of displaying an image on a portable display apparatus thatoutputs at least body state data based on an attitude and/or a motion ofthe portable display apparatus body and touch position data based on atouch position on a touch panel provided on a surface of a displayscreen of the portable display apparatus, the game program causing thecomputer to execute: controlling, on the basis of the body state data,an action of a player object placed in a virtual world; determining, onthe basis of a position and/or an attitude of the player object, whetheror not the player object has received an attack from another object;generating, when the player object has received a predetermined attackfrom another object, an attack effect image representing an effect ofthe predetermined attack; generating a first image by superimposing theattack effect image on an image of the virtual world viewed from theplayer object, or on an image of the virtual world including at leastthe player object; displaying the first image on the portable displayapparatus; and repairing, when the attack effect image is displayed onthe display screen, the effect of the predetermined attack in an area inthe attack effect image in accordance with a touch operation performedon the touch panel, the area overlapping a predetermined range whosecenter is the touch position indicated by the touch position data. 2.The computer-readable storage medium having stored thereon the gameprogram according to claim 1, the game program further causing thecomputer to execute calculating an attitude and/or a motion of theportable display apparatus on the basis of the body state data, whereinat least one of a direction, the attitude, and the position of theplayer object in the virtual world is controlled on the basis of thecalculated attitude and/or motion of the portable display apparatus. 3.The computer-readable storage medium having stored thereon the gameprogram according to claim 2, wherein when the player object and anotherobject have collided with each other in the virtual world, it isdetermined, on the basis of the direction, the attitude, and theposition of the player object in the virtual world, that the playerobject has received an attack.
 4. The computer-readable storage mediumhaving stored thereon the game program according to claim 3, whereinwhen a specific portion of the player object has collided with anotherobject in the virtual world, it is determined, on the basis of thedirection, the attitude, and the position of the player object in thevirtual world, that the player object has received an attack, and whenthe player object has received the attack at the specific portion, theattack effect image representing an effect of the attack is generated.5. The computer-readable storage medium having stored thereon the gameprogram according to claim 2, wherein the action of the player object iscontrolled such that the player object rotates or moves in accordancewith an angle by which a direction of the portable display apparatusbody is changed.
 6. The computer-readable storage medium having storedthereon the game program according to claim 5, wherein the action of theplayer object is controlled such that the greater a change in the angleby which the direction of the portable display apparatus body ischanged, the greater the player object rotates or moves.
 7. Thecomputer-readable storage medium having stored thereon the game programaccording to claim 2, wherein the attitude and/or the motion of theportable display apparatus are calculated with respect to apredetermined direction in real space, and on the basis of the attitudeand/or the motion of the portable display apparatus with respect to thepredetermined direction in real space, at least one of the direction,the attitude, and the position of the player object is controlled withrespect to a direction that corresponds to the predetermined directionand is set in the virtual world.
 8. The computer-readable storage mediumhaving stored thereon the game program according to claim 7, wherein theattitude and/or the motion of the portable display apparatus arecalculated with respect to a direction of gravity in real space, usingthe direction of gravity as the predetermined direction, and on thebasis of the attitude and/or the motion of the portable displayapparatus with respect to the direction of gravity in real space, atleast one of the direction, the attitude, and the position of the playerobject is controlled with respect to a direction corresponding to adirection of gravity set in the virtual world.
 9. The computer-readablestorage medium having stored thereon the game program according to claim8, wherein at least the attitude and/or the motion of the portabledisplay apparatus that are obtained by rotating the portable displayapparatus about the direction of gravity in real space are calculated,and on the basis of the attitude and/or the motion of the portabledisplay apparatus that are obtained by rotating the portable displayapparatus about the direction of gravity in real space, at least one ofthe direction, the attitude, and the position of the player object iscontrolled such that the player object rotates about the direction ofgravity set in the virtual world.
 10. The computer-readable storagemedium having stored thereon the game program according to claim 8,wherein at least the attitude and/or the motion of the portable displayapparatus that are obtained by swinging the portable display apparatusupward and downward about a horizontal direction perpendicular to thedirection of gravity in real space are calculated, and on the basis ofthe attitude and/or the motion of the portable display apparatus thatare obtained by swinging the portable display apparatus upward anddownward about the horizontal direction in real space, at least one ofthe direction, the attitude, and the position of the player object iscontrolled such that the player object rotates about a horizontaldirection set in the virtual world.
 11. The computer-readable storagemedium having stored thereon the game program according to claim 2,wherein at least the attitude and/or the motion of the portable displayapparatus that are obtained by rotating the portable display apparatusabout two axes orthogonal to a perspective direction of, andperpendicular to, the display screen of the portable display apparatusare calculated, and at least one of the direction, the attitude, and theposition of the player object is controlled such that in accordance withthe attitude and/or the motion of the portable display apparatus thatare obtained by rotating the portable display apparatus about the twoaxes, the player object rotates about two axes that correspond to thetwo axes in real space and are included in the virtual world.
 12. Thecomputer-readable storage medium having stored thereon the game programaccording to claim 11, wherein at least the attitude and/or the motionof the portable display apparatus that are obtained by rotating theportable display apparatus about an axis along a width direction of thedisplay screen and an axis along a height direction of the displayscreen are calculated, each axis being orthogonal to the perspectivedirection, and at least one of the direction, the attitude, and theposition of the player object is controlled such that: in accordancewith the attitude and/or the motion of the portable display apparatusthat are obtained by rotating the portable display apparatus about theaxis along the width direction, the player object rotates about ahorizontal axis in the virtual world; and in accordance with theattitude and/or the motion of the portable display apparatus that areobtained by rotating the portable display apparatus about the axis alongthe height direction, the player object rotates about a vertical axis inthe virtual world.
 13. The computer-readable storage medium havingstored thereon the game program according to claim 2, the game programfurther causing the computer to execute setting a first virtual camerafor generating an image of the virtual world viewed from the playerobject or an image of the virtual world including at least the playerobject, and controlling an attitude and/or a position of the firstvirtual camera on the basis of the calculated attitude and/or motion ofthe portable display apparatus, wherein the first image is generated bysuperimposing the attack effect image on the image of the virtual worldviewed from the first virtual camera.
 14. The computer-readable storagemedium having stored thereon the game program according to claim 13,wherein a direction of a line of sight of the first virtual camera iscontrolled so as to be the same as the controlled direction of theplayer object.
 15. The computer-readable storage medium having storedthereon the game program according to claim 2, wherein the portabledisplay apparatus includes at least one of a gyro sensor and anacceleration sensor, and the attitude and/or the motion of the portabledisplay apparatus are calculated on the basis of data output from the atleast one of the gyro sensor and the acceleration sensor.
 16. Thecomputer-readable storage medium having stored thereon the game programaccording to claim 1, wherein an image that hinders a field of viewtoward the virtual world when superimposed on the image of the virtualworld is generated as the attack effect image, and when the image issuperimposed on the image of the virtual world, the area in the attackeffect image that overlaps the predetermined range is repaired so thatthe field of view toward the virtual world is not hindered.
 17. Thecomputer-readable storage medium having stored thereon the game programaccording to claim 1, wherein image data indicating the first image isoutput to the portable display apparatus, the portable display apparatusincludes an image data acquisition unit that acquires the image dataoutput from the game apparatus, and a display screen of the portabledisplay apparatus displays the first image indicated by the image dataacquired by the image data acquisition unit.
 18. The computer-readablestorage medium having stored thereon the game program according to claim17, the game program further causing the computer to execute generatingcompression image data by compressing the image data indicating thefirst image, wherein the generated compression image data is output tothe portable display apparatus, the image data acquisition unit acquiresthe compression image data output from the game apparatus, the portabledisplay apparatus further includes a display image decompression unitthat decompresses the compression image data to obtain the image dataindicating the first image, and the display screen of the portabledisplay apparatus displays the first image indicated by the image datathat has been acquired by the image data acquisition unit and has beendecompressed by the display image decompression unit.
 19. Thecomputer-readable storage medium having stored thereon the game programaccording to claim 1, wherein besides the first image, a second imagerepresenting the virtual world is further displayed on another displayapparatus connected to the game apparatus.
 20. The computer-readablestorage medium having stored thereon the game program according to claim19, the game program further causing the computer to execute generatingcompression image data by compressing the image data indicating thefirst image, wherein the generated compression image data is output tothe portable display apparatus, and, besides the compression image data,image data indicating the second image is output to said another displayapparatus without being compressed, and the portable display apparatusincludes: an image data acquisition unit that acquires the compressionimage data output from the game apparatus; and a display imagedecompression unit that decompresses the compression image data toobtain the image data indicating the first image, wherein a displayscreen of the portable display apparatus displays the first imageindicated by the image data that has been acquired by the image dataacquisition unit and has been decompressed by the display imagedecompression unit.
 21. The computer-readable storage medium havingstored thereon the game program according to claim 19, wherein an imageincluding the player object in the virtual world viewed from a point ofview different from a point of view toward the virtual world forgenerating the first image is displayed as the second image on saidanother display apparatus.
 22. The computer-readable storage mediumhaving stored thereon the game program according to claim 19, wherein apoint of view toward the virtual world for generating the second imageis set at a position further away from the player object than a point ofview toward the virtual world for generating the first image is from theplayer object, and a range wider than a range of the virtual worldrepresented by the first image is displayed as the second image on saidanother display apparatus.
 23. The computer-readable storage mediumhaving stored thereon the game program according to claim 19, wherein apoint of view for generating the second image is set at a position ofviewing from a bird's-eye view the player object in the virtual world,and an image obtained by viewing from a bird's-eye view the playerobject placed in the virtual world is displayed as the second image onsaid another display apparatus.
 24. A game apparatus capable ofdisplaying an image on a portable display apparatus that outputs atleast body state data based on an attitude and/or a motion of theportable display apparatus body and touch position data based on a touchposition on a touch panel provided on a surface of a display screen ofthe portable display apparatus, the game apparatus comprising: a playerobject action control unit that controls, on the basis of the body statedata, an action of a player object placed in a virtual world; an attackpresence/absence determination unit that determines, on the basis of aposition and/or an attitude of the player object, whether or not theplayer object has received an attack from another object; an attackeffect image generation unit that generates, when the player object hasreceived a predetermined attack from another object, an attack effectimage representing an effect of the predetermined attack; a first imagegeneration unit that generates a first image by superimposing the attackeffect image on an image of the virtual world viewed from the playerobject, or on an image of the virtual world including at least theplayer object; a display control unit that displays the first image onthe portable display apparatus; and an attack effect image repair unitthat repairs, when the attack effect image is displayed on the displayscreen, the effect of the predetermined attack in an area in the attackeffect image in accordance with a touch operation performed on the touchpanel, the area overlapping a predetermined range whose center is thetouch position indicated by the touch position data.
 25. A game systemincluding a plurality of apparatuses configured to communicate with eachother, the game system capable of displaying an image on a portabledisplay apparatus that outputs at least body state data based on anattitude and/or a motion of the portable display apparatus body andtouch position data based on a touch position on a touch panel providedon a surface of a display screen of the portable display apparatus, thegame system comprising: a player object action control unit thatcontrols, on the basis of the body state data, an action of a playerobject placed in a virtual world; an attack presence/absencedetermination unit that determines, on the basis of a position and/or anattitude of the player object, whether or not the player object hasreceived an attack from another object; an attack effect imagegeneration unit that generates, when the player object has received apredetermined attack from another object, an attack effect imagerepresenting an effect of the predetermined attack; a first imagegeneration unit that generates a first image by superimposing the attackeffect image on an image of the virtual world viewed from the playerobject, or on an image of the virtual world including at least theplayer object; a display control unit that displays the first image onthe portable display apparatus; and an attack effect image repair unitthat repairs, when the attack effect image is displayed on the displayscreen, the effect of the predetermined attack in an area in the attackeffect image in accordance with a touch operation performed on the touchpanel, the area overlapping a predetermined range whose center is thetouch position indicated by the touch position data.
 26. A gameprocessing method performed by a processor or a cooperation of aplurality of processors included in a game system including at least oneinformation processing apparatus capable of displaying an image on aportable display apparatus that outputs at least body state data basedon an attitude and/or a motion of the portable display apparatus bodyand touch position data based on a touch position on a touch panelprovided on a surface of a display screen of the portable displayapparatus, the game processing method comprising: controlling, on thebasis of the body state data, an action of a player object placed in avirtual world; determining, on the basis of a position and/or anattitude of the player object, whether or not the player object hasreceived an attack from another object; generating, when the playerobject has received a predetermined attack from another object, anattack effect image representing an effect of the predetermined attack;generating a first image by superimposing the attack effect image on animage of the virtual world viewed from the player object, or on an imageof the virtual world including at least the player object; displayingthe first image on the portable display apparatus; and repairing, whenthe attack effect image is displayed on the display screen, the effectof the predetermined attack in an area in the attack effect image inaccordance with a touch operation performed on the touch panel, the areaoverlapping a predetermined range whose center is the touch positionindicated by the touch position data.